Country
3979818cdc7bc8dbeec87170c11ee340-Paper-Conference.pdf
Self-supervised large language models have demonstrated the ability to perform various tasks via in-context learning, but little is known about where the model locates the task with respect to prompt instructions and demonstration examples. In this work, we attempt to characterize the region where large language models transition from recognizing the task to performing the task.
Wefirstperformsecond-orderexpansionsfor`andQalongthesolutionpath,whicharegivenby ` `(t,xt,ut) +` xTδxt +` uTδut + 1 2 δxt δut
Forinstance,consider the propagation of afully-connected layer,i.e. Let u t u (t) be a solution that achieved the minimumof(18). In our case, we consider the linear ODE presentedin(22). Let us start from the terminal condition in (29). With these, we arrive at the second-order feedbackpolicypresentedin(16).