From 79e84fade3c97fb485e8a3d7f4e3da0df4d44aa2 Mon Sep 17 00:00:00 2001 From: YWMditto <46778265+YWMditto@users.noreply.github.com> Date: Thu, 9 Nov 2023 13:12:38 +0800 Subject: [PATCH] feat(doc): add dynamic ntk example (#480) * add dynamic ntk compare example * add dynamic ntk compare example --------- Co-authored-by: YWMditto <862779238@qq.com> --- doc/aux_ materials/long_text_example.txt | 1 + doc/imgs/dynamic_ntk_answer.png | Bin 0 -> 32528 bytes doc/usage.md | 36 +++++++---------------- 3 files changed, 11 insertions(+), 26 deletions(-) create mode 100644 doc/aux_ materials/long_text_example.txt create mode 100644 doc/imgs/dynamic_ntk_answer.png diff --git a/doc/aux_ materials/long_text_example.txt b/doc/aux_ materials/long_text_example.txt new file mode 100644 index 0000000..7fdfe2d --- /dev/null +++ b/doc/aux_ materials/long_text_example.txt @@ -0,0 +1 @@ +Please summarize the following passage:\n\nBy Alex Acquisto , BDN Staff • June 14, 2017 2:30 pm\n\nUpdated: June 22, 2017 1:24 pm\n\nHOPE, Maine — While jogging on a familiar, overgrown, wooded trail near her home on a recent warm afternoon, Rachel Borch thought to herself, “what a beautiful day.”\n\nLittle did she know she was about to be attacked by a rabid raccoon she would end up killing with her bare hands.\n\nIn the midst of appreciating the weather and scenery, she looked ahead and noticed a raccoon obstructing the narrow foot path, baring its tiny teeth.\n\nSuddenly, it began “bounding” toward her, Borch recalled Wednesday afternoon during an interview at her home on Hatchet Mountain Road in Hope.\n\n“I knew instantly it had to be rabid,” said Borch, who remembers ripping out her headphones and dropping her phone on the ground.\n\nWhat felt like a split second later, the furry animal was at her feet. Borch said she was “dancing around it,” trying to figure out what to do.\n\n“Imagine the Tasmanian devil,” she said. “It was terrifying.”\n\nThe path was too narrow for Borch to run past the raccoon, which had begun lunging at her. With adrenaline pumping, Borch suspended her disbelief.\n\n“I knew it was going to bite me,” she said.\n\nFiguring she would have the greatest ability to defend herself if she used her hands to hold it down, she decided that probably would be the best place for the aggressive animal to latch on.\n\nThe raccoon sank its teeth into Borch’s thumb and “wouldn’t let go.” Its paws were scratching her arms and legs wildly as Borch screamed and cried.\n\nIn a matter of seconds, Borch, who could not unhinge the raccoon’s jaw to shake it off her hand, noticed that when she had dropped her phone, it had fallen into a puddle in the path and was fully submerged.\n\n“I didn’t think I could strangle [the raccoon] with my bare hands,” she remembers thinking, but holding it under the water might do the trick.\n\nConnecting the dots quickly, Borch, then on her knees, dragged the still biting raccoon, which was scratching frantically at her hand and arms, into the puddle.\n\n“With my thumb in its mouth, I just pushed its head down into the muck,” Borch said.\n\nWith the animal belly-up, she held its head under water. “It was still struggling and clawing at my arms. It wouldn’t let go of my thumb,” she said.\n\nBorch said she held it there for what felt like an eternity until finally it stopped struggling and “its arms sort of of fell to the side, its chest still heaving really slowly.”\n\nHyperventilating and in hysterics, she pulled her thumb out of the raccoon’s mouth, “and then I just bolted as fast as I could through the underbrush,” she said.\n\nBorch remembers looking back once to see if the raccoon had started chasing her again.\n\n“It felt like [Stephen King’s] ‘Pet Sematary,’” she said.\n\nKicking her shoes off because they were soaked, Borch ran the three-quarters of a mile home to her house.\n\nBorch, who was screaming and unsure of how rabies affects humans, remembers thinking, “Oh, God, what if I just start foaming at the mouth and can’t find my way back?”\n\nShe met her mother, Elizabeth, at home, and together they drove immediately to Pen Bay Medical Center.\n\nThe dead raccoon was retrieved by Borch’s dad, who packed it into a Taste of the Wild dog food bag and handed it over to the Maine Warden Service.\n\nHope Animal Control Officer Heidi Blood confirmed Wednesday that the dead raccoon later tested positive for rabies by the Maine Center for Disease Control.\n\n“Not to scare people,” Blood said, but “when there’s one [infected], there’s typically another.”\n\nIt’s important to “let folks know that just because there’s one [infected] and it’s gone now, doesn’t mean the risk still isn’t there,” she said.\n\nInfected animals typically start showing signs within two weeks, Blood said. Humans can start exhibiting symptoms within a few weeks, she said, but often it takes a few months.\n\n“It’s scary stuff,” Blood said. “The No. 1 thing we try to remind people of is that it’s 100 percent fatal [if it goes untreated].”\n\nBorch has received six shots so far, including the rabies vaccine, and immunoglobulin and tetanus injections. She is slated to receive her last injection this weekend.\n\n“If there hadn’t been water on the ground, I don’t know what I would have done,” Borch said of drowning the animal. “It really was just dumb luck. I’ve never killed an animal with my bare hands. I’m a vegetarian. It was self-defense.”\n\nHer advice for others who find themselves facing a rabid animal? Borch said she has none.\n\n“I always thought of raccoons as this cute, cuddly forest animal,” she said. “I just will never look at them the same way.”\n\nBorch is not the only person to have been attacked by a rabid animal so far this season.\n\nEarlier this week, a Wiscasset man was bitten on both hands in Topsham by what was believed to be a rabid fox.\n\nAs of June 7, according to the Maine CDC, there have been 20 animals, including raccoons, red foxes and skunks, that have tested positive for rabies in 2017.\n\nIn 2016, 64 animals in Maine tested positive for rabies, according to CDC data.\n\nLoading… Passage 2: HOPE — A woman who was attacked by a rabid raccoon while running in the woods near her home drowned the animal in a puddle after it latched onto her thumb.\n\nRachel Borch, 21, said she was getting ready to go out for an afternoon run on June 3 when her brother Chris told her to be careful because he had seen a raccoon “skulking” around the yard. He told her he thought it odd that the raccoon, normally a nocturnal animal, was out in the daytime.\n\nBorch said she was running on a fire road next to her house when she encountered the animal.\n\n“Out of nowhere, I see through the underbrush a very ferocious-looking raccoon charging at me with its teeth bared,” Borch said.\n\nShe immediately knew something was wrong with the animal by the way it was acting.\n\n“It was one of those moments like out of the ‘Twilight Zone’ – this isn’t real, this doesn’t happen in real life, but then it was right there and it was right at my feet,” she said.\n\nBorch said she started dancing around the animal, frantically trying to figure out what to do.\n\n“There was nothing I could do, it was going to bite me,” she said. She had dropped her phone and had nothing to protect herself with.\n\nShe figured that if it was going to bite her, it might as well be her hands. She put her hands out in front of her, and the raccoon latched onto her thumb.\n\n“I was screaming and crying and trying to hold it down,” she said. “There was a few inches of really muddy water on the ground – it was a swampy area of the trail – so I just took all my strength and pushed it into the water.”\n\nShe said she pushed the raccoon’s head underwater and held it there for what seemed like a very long time as it clawed her arms and continued to bite down on her thumb.\n\n“It happened so fast, but also in slow motion,” Borch said.\n\nThe state’s Center for Disease Control and Prevention confirmed June 5 that the raccoon tested positive for rabies.\n\nRabies is caused by a virus, affects the brain and spinal cord and can cause death if left untreated, according to the Maine CDC. Rabies in people is very rare in the United States, but rabies in animals – especially wildlife – is common in most parts of the country, including Maine.\n\nThe raccoon continued to move for a while while Borch kept its head underwater. She said she was afraid to let go for fear it would continue attacking her. Eventually, it released its paws and its jaws stopped clenching her thumb.\n\nBleeding and crying, Borch ran home and screamed for her mother to call 911. Borch’s father, Brad, and her brother retrieved the animal so another animal wouldn’t drag it off and become infected.\n\nHope animal control officer Heidi Blood said that if one animal tests positive for rabies, it’s “almost 100 percent” certain there are other animals in the area also infected. Blood said that last summer, two raccoons in Lincolnville and one in Hope tested positive for rabies.\n\nBlood said an animal doesn’t have to be aggressive to be infected; it can also act “delirious or drunklike.”\n\nShe said people and their pets should avoid such animals, even dead ones, because the rabies virus can live outside the body for 24 hours.\n\nUnder Maine law, cats and dogs, even if they stay indoors, must be vaccinated against the disease.\n\nBorch is expected to receive the last of four rabies shots on Saturday, and has been taking antibiotics for the puncture wounds on her hands.\n\n“You just can’t predict something like that,” Borch said. “I’m still processing it, but that does not happen and that is not a normal thing.”\n\nShare\n\nWant the news vital to Maine? Our daily headlines email is delivered each morning. Email *\n\nNewsletter Choices * Daily Headlines and Evening Express Breaking News Business Headlines Maine Cannabis Report High School Sports Real Estate\n\n* I understand the Terms of Service.\n\nName This field is for validation purposes and should be left unchanged.\n\nThis iframe contains the logic required to handle Ajax powered Gravity Forms. diff --git a/doc/imgs/dynamic_ntk_answer.png b/doc/imgs/dynamic_ntk_answer.png new file mode 100644 index 0000000000000000000000000000000000000000..c582cfebbd7ccf0d983cee0c7b0fc9fc17229be7 GIT binary patch literal 32528 zcmeFZWmFu&wt$PfyK8W_U;_ie-90!1!QEvbSa63B++BjZd$1tE-QC?DIrrRi&pr43 zegEF9)zj5owX3Fg)$Xd^`|Al)R+K_TB0_?IfIyXz7FUIUfc~3~LV$b!#s@_$LqI?R zEycu?WyHiNl%4F&Ep0#$5cF}TMn;=5%=AOX#zsa%V@wQ4PVTDTzD24UfycYXy2iT( zv&Xv9zJLC_PJpovt=RL!-2je`$K&$jJV()zB(SL&?1qZ4@!(q zqnnFOM@g!CsB7qZ03^;IjM*%E|K3=YNNCFBsqrqOa#)=0Pv1a7#zym883Cf_IG^N& zi1$ea;XTf|&H8B~P_r02_69{DI{Keh{+FEbyy{bQ!>e?>a3i1M` z_O`6XX7(l^Ru5Z;zakI-4}tfjEy%@~!o$|a&RM`ii0U6D1m4qsPqR@`{G*7AwGfrI zf-;4ey%UJyBP%;AJC!gJ1qB7b$;@0pRb29);_pX7R9{?N90b_d+}+(--MLupoh;Zm z`1$$S*g4raIa%IIusD0#xfpw}*f~@GS0(>lk2uKL)XCDp#nRr6;%~jiCibo_LR3_L z8~WGhzkVmk!}5PxvUC3DWxZb@+us~E4pw%yf7Sgi3ix|gK-tm*WTPW)Y5T68_cnw% zxH$m-DF1(Q{-?!%lhpnnNnT#Qf0z8Xod1;6a0WSv+1tK1=_33;J@ZfDf6x4#$w*MBek&n$5<5ha{l(OlC1mm1$iIRAfmbI#6y0u(`%dL? zxV>eeoGX4T9rxBTq}Mi?|DW9{y^gTljSU<5oa%$=^0-YS;BwBc-{BRfS|BU4KsNGe zIG$!F#VQxW6Z!4`4ik3>DEDJ=$mt;KxytEW*v?6Euub-8D%bSNU4H&_kk?h)JYRv$`-r1L#2 zlj3{a%-w%j1R{9=47bWUA{6>7wYJ8P`?Nqc?-|_sGNkv;Lr($nUWvy~I{IIMN-CQl zWIO~M@IOs2aP57$Jw7`KVS_#6{%|V?nFWfaMt-Gx`9CHlJ&B+#HPh&5{(=o*w!Z%x zyUu5UvBeJ+FVe^5YFFX4ll4!eLvRxH^PV3!uM$lB@8q%8gOq}!FRN`5b(UKnZ&SXz z<8rDjdaF&Le&UlSe@>1y!Ok`WzeEkR3vztSyI1mjd%05-0KfKte}v3OJ|8bQ*O~5B z&PP%aor#4;PsUNdNjjsah}m2<7CKL4wsTuud(YfPjn^4(nI9ch1JbTswgym%gv_BT zUbd718z0&?itSIeVenXVWy$@nWg;)h(Z~$2jFR;Q7jAo~SznHq+(i!8I^W2m>c7~- zlbEnv{rD+Kfr9WMn-um5;@(lce5zA>QoVT$XmIH{@~dZNA1xUGn;dec5eX`)wK{G)k*@R;aT!T39w`H<`r5 zyFXPR=W?+r(c+cVNaS$p|6&3~vAvZ6T})^U4}}f2>w36N_yhQculv)7C5cfZs8q{4 zInbCYKt?KGHXi2owbtiBmQbEbJn+$VzvJojhKw&n`0ZCyt(l*{f&J1~*(B!E9}zTi z;2c)_yTUPIt=7jVB{FuzkyI|hf|y>;lbH)w*?2x~o$IgqHdnkw#)4;j83504IP^m? zP$nbD5BHCEBcQn*-P(k^R*)>f6oBDXc^ScNz3bw2{aeZ;C^UJ6pt}fqPz;8M$4(-O zc(&+~trG`b&E4VyL$^GR4`9F3u~#a~M<8%Kqg{G|d}JiSKmS=F=(x^#5C{-$61TWN zhIB)E-b3@~nn;cw#p$+^{7!Mti{>#EpbvBFklD+sT^D)#TpM8k&V!|qC21M>qjZHg zd8NVhpp}`zoPz7-Ok{HP+8HT`m<4G$-I7{#Y`nMSmY*wujQP41Z`GmAvb~?Jioa2# zfQS9sO?!-^ge8Iwz-SeevGa8DCb#LRds`w75Wh#VKp75`;#(4AY2{$1Ck3ohIvgf; z8h&#}Yz@T1xx!=5^zVOy2nwEkys=h`qw@SRSE*m;d8MiQwg>n6O_y<1qA79=ijv%o z+)PU(TKH*2M<1~jg)^>;H3<`Iu{ZEbD{o;eJr&#XywzisDSe>v#~~}&BD~R z{6f)wQ8Q4^E8S1??N+efY9;P+myMxRkZ%y;@pg&aBdwX?WDTsG`BM%Gj19ck&a40$ zY`$JiM9p;cE@5fyl-64w)=g~tz{4d@7y2sBmd+Jm6=W9J2|fPK?mnB`MlT}#-S+#G z<T@PN!{aR&sA{UoBf4r}yBx1s{#oHn>3L{9;`~08 z5eq@P&T?t^{OtuD$e^iGDUDyP?6T1#v0G*- zqH_$RUGjX+fiCm(uW-n2G%ql3we$Co+h}J}(Oz2ZHSJV}=Yaxn&v9<$jDhAp4^cz0 z@t#eTS;cg*Y#6KUTu$jm;rAFoGm?Dv03T>~qmC~ZoczT3?4JnA(nVL{QHVt4QYvD- z(0D{A1EApxI;c^$EuXy;P;?$a{fryV=aa@45#)))hoMv`Uor6y17w#bjL}9ph!qFIKU4c zaA1S&n_fF-Wxyh6ryCjlUcb$Z-}Piw>udW;MNB>^V=)qwGe4^t|20(v-_}+z1+SfL za6Zt@GoQBMEe7-K&|d)ab|qM8kd;GLS6H4CuEBa zF~YCvQ@6Nxw&q9;Rqnmi7;yCNeJ>kQ)A=YW(Xldh_NDp~PhXPsoOSK@o?y6CzI_wv zZ! zN;9U^21Gq!=V+Q85(1y(G&9s)j(3A)yBh+1-JQ?C?R4$obvca1ZRTM3J5PdUq1C;! zZE(VH>9w#oG=^ZJX`#)Ni4^Fly3D@Nbv2w&v_tsneMk6#>3q6SF~27mX|g*ov{?Ih zPi_aa(#_#aB6xp2&lC=CIDxL{ZjD$1_HwU)ObA|DiVj5qn5r{|sJ;Ff1|bRGapHLw z98$H@`OJ6^xuoCvy3swbr=H`(|AXD{!_HV*=;ICfh||{E!gro0!BqM>yV=BnXhK#& z**zOH5IN836D6AHv-%AXLwTcUq=nd2}nQCo-dXw5Z5*AQJ`YH!gdcx6?fHEVK_~9I$z}C@?U#SF9V3@y zzbEfbNI&a*3lr#Ehz9=A9`>+k7v1|7z-{)E=BVxN_#!#KN@lvji50#Ep77CimWXXJ+dta+WDdSDL`Xp_j~${flP?zECxExLY;f z9CF>Xxsz2&`=#=;)2_!5AFEG8)~U`PWm`mT>|iE4!Un|>^$a;{jWk|e+Dd9nJk|R+%cqbSzx7>>$K(+zq)NVpxFMSNvHUxRqun(YH+*Kba18L>9*2nc3`!(PMPPk=5tkhHpUD8u;@y#ZoQV%X+Jv8 z&~b%d^;EfL_I{%~{CY+d!(sP`M#i_9b~`M;cH4P(Z#R@?{ruzX-hQq!u%pr*{jR-x zW66DguhnIz^d14PDY-)6{zsz?U3b&(3!%8Gpct)Tl#hU7EZ!Z6H`l|DsxiWL1it-p z$yXO$`pX{dl&_wJOsC{Z4>RZ1?T!(|XIWL6gM)i~xiCAkE@6T2J1WZ?bH}$f!H2?? z(jnw_A|FW5yZTQKHGo+Y!f(2n&Bh6L%j|^4Gh_3P*gF|s1Gr6;XX|Cs%{kAb`PDnP z6pp*1>F5v8tII|a#99u?F>#XYMrG2$d!pfd7vTX_@(Uv!wH6bMYz=e6id2N( zInb`jad6Li@z`~1A)T&YY?ua+y7r!g1Bl&@IracIx5Ky$1gBbDiTlpxxH@^BFZx85 zJus+jG4DH9>SHd*F5!7&FI!y>*ao~;5#!H?B1X`+^@jcN`BiPv$rpLaPnVmQ z90Exws{qnCyw19wuFF~_vMdi$FK`#*p>b|W4aU8LLxRu6oKhX-FggJ?xqevySJ_Po zAviJ$gJ+CJa12>O8j9{GV^-d?s@kQO3{u4+j?P)Icq)g08ddS62A?b(zY;0% zWXYXi{I5xF7U1isp9E|gC)-j(bp$L6)Z^OKm@6n{KU$6j@}mNV6FGI89Ktrr7r{dn zBfUc^TTyp<4R#6aomKjDc6K*P7Y^})Y&DgK2ufdR4Epg;A{g;!>8^Ux?l+PgWR5Pj z=pwK4dSR*oy;~*#^=6_HX%NH;7crnHEm$NH9PIZrj~nw=k(p<77y9_nY!3656QJ`zIJLEdg^#a zIJrjX_0QG~m$}@nxJx0nnhq*@Rw{3#(w}np`fCuEQy9AvdqTizE7Vix+v8SMo8Z!^ zS@SC9JKt&n)e%<5rij*bBJG&MPQCk&-=Gs`HMMqH2DhP)K2CMtz^1bPKQ5jNV{Sd ze-urk77IbyJlz=#>(svwMoyXfv>FyjgrNCHWZ9^;G~4^dkr5wPE15ZUmPII?sc6~a zW70%kGdX2_p1B9gCY&UOUpiNo=D8gLx}w<}%hqkNhtMxhP;$4(F|W;eS&xyHib{u% zCxR`cdpXq>kF_SaTbF755b(rWd zBjvAA=|6I&9Xh-ldmpmPbPOqw@@{^%C!7_boo$~zEu@g@)tl{RLgut>s0B#7`BTEH zxnGg@F;$+Bi^w*$lqtTU=YwEC@zT7G28{Ect})qno7yR)N3W`3N;2CitlVeIvEmM0 zV*7#$AvJjlzEzHMflX?ppIcm>V021=tka=bx2I{WF0F84#Ai}Ea<2Am*8+?Z6!Qh} zkB|gqh$ts^>=}dCpjnWPW9dG~vSY&<_DPJq%!~PuQNn^oUC7bt?I#~y!540xwmF+9`#7*G;{paw~&2+j#CH4Vw7M95R3-ZscjU@(TD9h9;&m7okY<(On>|3&5%dbeWD{G-2>(_+W?FdV$X{_4+=vr;jB-{c+7ch1xDx(BHiBh{??_?#t+y`}u@zG2RkOgdRY3O{Bx^;3I;45(3WmdYo_P(wRa-? zMPF-dmy01UWcd_4-o%vJmPh)dWvJ4Ir8Wr&$h!!W2NZOF@#CWWX{|=WouN>FSS;us zMg^g?&^=Tj@(Zw`SS68&6+mDI<1%UeUO65wvx0GD#()c_u<@mnb4yvy^QG4;|5k^) z*#*Qbgl@_i!ED&5phc{3tyC}Bboz*xU?u-~)nt~*Xu9zYL!~Updr#56*i52zF)8Iz z{lnDf6LN5Qk!^YDw_5$;+VAt9Vu%idi>I@h%BkQ~4-(*CWxBCcNT% z8H{%#f9DoPTZ777CzM9!mi{wZYlEXAesnv>&2~?Y{Yx6KZ@2&CGP;{*8MM%?haKCo z&5QQVctrUUN6lrxVpJ;Y`6raa2L6+##u0{|pmaF7^xgX$M%b6f7FTPPs!aWTz}+67 z*!qlCM7Y}rY+gd;GmUok!HU)9dW~InO~BY0)C&CcoOge3WPpBK7?ySMLnmOk=b)6XC1zJ~kpX-tTW{0LsLZe?(coaHsg3$;o@ zm%NOM#CHL9r!7pWE@;W?gCNU?5>nh3}^LYE?Ap$Gm$*+=XTPqlXpawzzhFqZcM#;)2veZDZ{+Au#Lr!5RH}_9n z?t!x^8wJcz#psH2bgJETU*B-_i}iaJ>Dxy|+h3(;@%tug?alqf%?9rWFyE=Z1qjhw zH@I+n^{KBCjZ%fCPh|G4&r0z-3H87;TKrBY$2&K5L6|H_-wI@aenQ`Zmfzz;p@qh* z1FtE1<>_0mzU~Hgq&0X-M&4jQoNwv!Yg?fO*)pM+fHxrp_A{mHv#{gf{f&7E$~O3} zBaqOndCCXrnIZ|kp_2eU!TBi1SkBgjX{!d1b9qL!vVx%@0&&$poOk(WJEvqu>cbIs zRg=~Awck{o)O-qXNiE?fcabE^(G0Gx)?61apBRJ}8I<|_bW0v9I`DEZ{mV*qbx83I zp3Lms6KS9V^0&f*qat{w_N{&l>V|ToI9Aub-1Yqor^AO(WOIqXBgz5Co~4*23K{`e zeqdV)1jKE(nF`|R)shOha;+K{z%a4%wV~q1kP5nAx!pUGRWC0-@&NSSS387<*fO49 zA$V$v_!aKXLqVfuTy|$HDlNjB)oc!Zc7L34f2nNrzmLKM2ilRbeE6R^JqIVKJ!~F0;hE5@$L1Z?{KE(}7cS!Vo zWaw?t%lYc(N>e`h351Lc-Gy=jSN@jWA@+iiU+vsO=8A`nLBKWt?*C!y@L_OWOP4Ql zBYbT;XTS1({wcMC39m|NI?rF7X9W7K&~UQJ-pbGhHLUx2MoAve+VMdY7S$Fw;)B-+E{Hc6>tk{i-?EVO>9WksLJl^8%@H|+e>tpaY^Ev=##eF z=6VG88FBF^s}~oeS_?td_~rU)N+gS8x-m~Y-GSxlC>pu&aC2*fYN7NfZX*r4&D=#P zsb-9J#V#!;+RYDZm(yogA|5NwBxcR@3LL;MyUTd3`q#}fPP5!!S{^;!Yc}7EZfMx@ zH9K}iT|jIjKpF)|ntb{j+{FZ#72IrQxBWt~fJKM9qv4=SZ+|1SUGFPqegdjywqe&1^@ZKftFPq)2YMAMprlpGCm~Z<3`?=kJ|3~J8zDCPlYTu|OtCw^fW56K&?R0` zUVWhqC0;j-HLl^1f;}k29Fm$P(7yddnsS-p&tFn%3Zavz>!DO@#5ux{fqYN?bsxXX z%gayZLUW&~;+P)sE5ioJaNh^4CA0h>*nnQ~kh!|j$TB6pmcU|jOTmyj;G76cl|6U$ zjwYf~t9k=t1aBBV{-&*qp6X3#qFKK^7JOjN3?QPwW* zv##C|NJT~sF8_Z1+lw%JJcHaf*FS7sDq%1Jjft7-B1ikR3gTJs+FW~-sbZbkR< zr(_-R{Eo;{!8nUYH(6=u7k6VM>_Y9%N2`W&b;tF?8P=_Ke6sT*gom%NOm()g94Ix% z)kvCnCuVy6VPzV2V`mQUE1jGfC2JDQxR*fN_9%7g64gy}T=5T< zIM@8MGU|y?_+L-SeV`-`o@kCDvEgbB+EG8dCGS5Pa0SMf z2jt|Z%37TdZLN+gdrO^f#Z{(*uN^|(F+I=7TU~F2>FtZMg}~eN^5~^U~ zH~h1V5r{v&^vC6~( zv|1W+wzjrMV||U`2(33wu;Px%4e=otGadtzs2jjzP=;B0p}p4mXd+oG(Ur6H&OuDS zam1aX)9tj&_3ah9@`i=P35E<{`BP4^%{dT+HB^t?Y#eh}?zGeQX_deJmw{)o^kf4G zpNI4!-;TLEswUow*%B*SSz1DQzr>5U*W@lW+fQ2!+6FVy9k^8k`gBVpT!50(PBJSU zp@_Ejd?0gnXKlQmb{Y7HYtj6V3Bo*3??q536dApU4{Na%##ypkK<#QmlY0K6kIbr< zU^iaOsfF*GT1`(Bk6rrBuXq}vq^fgng^WwYn}cZ;H-wM&k7?25z>z&-gDnlN;}0LO zl7h;#>?cvbe5uB6)G9~oEhEGnF8@7nPsru?tS6I!Tip`+MA($7beZ;$=E^+g)oKo` zj>kgbKNu+5gSOFhlFgd6ubz-aUt%TMboGj1pMU83Sds));7MHO7(lj7iW1B+^Re=C zJ7Y5U@~Ym}#bsMEdWZ=f(q+D;splh&=yXx%9CAeV3k4aj)?@&`Kk;eKd0mKOt#%L> zqd}O$!!R#m$Lk^XAjNCT>3;tO4SPRpE^)9}bfQm7KQ8&I>D^F8yGyUIvY^damKwmo zlRk4^N4b@DWbBkEl9{S4&1R|J*ws0{zNt7%8r@C0Cl23wwS3~;)!v<`$2aY-AMK#M z;TR8Tgn-*CVijU%Tmdrj9e8LDm6y~nKMV9k!zb(R^K5-msK$o7I9p`)1EPD7 zGFPui(bi-G40hIc4_L1=9%R(OzzuVvQS@jEqPYq67%HKL7>z+{e*1XIk2^%_GGPpf z*pfxjc-P%m@Vr@;n)cBy&cm1p-%@fh$*cSo6b&wGf zT{R9%Nm*5sNj7tnHB(Z29D@q}?K}NCIbH%5onp7)y?(!8gOI2K(VT4c*KDrB{{*Pm zsbl@=kX{3a7;E{e4z}2spFDKW$9RELol?=N6^lF!Vgq&096UTK=Y||IM!yK_+}}7} z|MUR7_H!al^3o=1e;_<>VD$gKHdwRzCwccT&d$yd2J2iLGT13_K9@X0$T_al?~(Up zr7=b$rgELTmPB&3#!?WBy}=3cJ{KKHr=iySWb(AP$98px7d;gx_NE%1c&9v`-t6Jz zsCHZ{hRR@lJDQu4^-&{Uns_A4er;-wVeUVHG#C+z4MiNP36e;%7s?D_*J!e#^nj`J zZUvw8Kwtz#sQ7ybi0_Q~`4^*%@G6H#+~wYc6gKVopmH=RHka$2MNvJ><`2lJ$L~cRG|Dv|Oz}Q*9U{!jm6LwwMP;SBr4K$pn z72WE^b$5}KO7Nxq#eVKrdY|=!j%*;!g~r}IS#jxOvo3LmPTn{RW0ctZW3(kMRZ7*v z7;GPBT(-}RSAC6UsU7FIf|r`1_Uo+ww_@k1V;$&_;vV|#x;xt8_m`U^T&_aE@%~x_ z`$aO|=qq5b&X3Y~_CpIHG{1_aP^&@@>890kny(1u~w^i%bMO2-)-xW$nnNR86LG;eRHTy#cN7-VFmrM>0~dEcs)&_$?) zpd4&NIGg+u7fDs<$2IhcmdqoSV1{|_oJN(4%jqFV_^S&?8~O1$0{X>JI7|Ly!~ zBMcHO+6`uf9G~Q^i4|gtpnEC)PpfDe!)&5Ll;&}*5nf(*FVT>CmU)$F`C|KYaMD6oI%t-}{SKkD&rEstX}K^dAZ|COZ> zM5^}^N?^WlnUY=+iOL^t`zn2vBPO+vpqDp;1IB2grbW7dx7$lZA?HY@ZF{2xO}t_h zfYd7K6w|lG=J(Q}S19dsK79lox(Jika2afS^bU-9DO;&wSxjW+sTC@mSCO zzMklaeED|jPhg_E#%kC>Vv*6FG5R+^Z1u-(XswSQi>UyV0+h#?l+1XJ*#>HI5=j>R zd1WCfKN+DU@fiagZ+^4X8Pt5c=#9B5Hh7`6>2{3cM>cM4+yY-=?gPD=&_Fwm z&QyYtrLZ=0Nj9mX9gHB!XykAT@Wu@D-VwLp2oR32+cWVyp9Os8;|&K^p}V;@stwMS zm_B_p(D?#(^cR6@^umEh2w@0+_YF=3BjYnT@4}-*7fqydoZdu9uDgk>j~=`!nj0I5 zs6C57Fa9==CbU*L1B?9^y+1+0B_@)T(cQEiBgP@VmJ=O!{XL5Xx zVi%f+J4oN&J0O@OOMj)}!K6O+;lTfAp#Q0kDg4}*&p&-17WNE(mqqERu0GS0`1IrN zr|GPPb#hsZtCoEv-S(T>F1dCJEWGTro&#PF7a0z+mVS1Sgo=BI<>hKVFvUV38eIuWi`szH>AW^*&=> z+r{oBPP2>G*IXLK?rYsPinn+%Eak*u_c_Fo=j+g&N3Y8m!RTZdXnt|R5?me6lFN#j zZL{YtCx2236`$liH`)xcDaE8@o`--v-w(P10)yB+;3I(#E^vDFmwFvYw7%xPX6rri}9N6|1s+7k4gRh^kE529ZF zwpo5+`J+i$@3ijk`#V~ds2?{wzDv9U)2A!m+dNa7>Ta=u+Y|DEaGhe*tArMO%>|Gb z723_=npFnWI|vWT+sTbOwVg%pQCIBrIK|1sH9xv_|F%C~2e@Ais#23#^h9G#+wA{5 zqb@eQ*k2vd-$})g<*DSYaaBVCZQhpRbQE#ksLM6Ka$)cRVHSGcCXMqDzB4)XtQ}^QkXZ9NGk27EQIhKT{6gPAF93l5ayExLGwe+O5#q3M~rEKo1o)>o@iS zlz{;r8D>znkKWj{W!t~Az;=mFoO5#NKaZX@m3q>M4lmi5 zGzo2Cx9w&R*DiF2N^Bh>dMnB`NCGNrn^)9MTq873EZ<%OL$n%ef^D>y6q44!uNQY} zhYKZ21KaC)F~Fn_q8fCc#ACxYk6~$aI~?G-l&;F4axy{;yP+dFY!9JaTcEtjXsyYP zl_T#POlj*7{-TwHqAUIZH26Bo{LyjT$C0;jV)yP*m=ES!oRY8Y(`}UK+l*zFTJYu0 zm!A8$L|%Kc!#Qwqx@cD;QK_bc=J&(d5{5Si`X0S+9<1v0og~U$3B((oSNJMd z=1~ZGE(BlD!4~B~62WWlex&yQ!BP|+u7oH9FH_f^W4vw%#WfhU^8Zx)n0n`%^2q&~ z>h-@xlS21$Dka5V>sx3PYZ29B_!&iy!$QlPTQ0P5+`LmU=U=AkSTdqNOGp+-JWryI zXFM4n{%+?2x1CZc{NUD@vUu&5I%U_<9tvcWSo3{L2r{hQMR)8GF&bC8dIq%X7yDu=i`Ws_Dpll?M{gL%zMc!ha882JVk8)Xvnn!ke;xg9O4! z0c%e!YYVI1N%jy;>)~waWb2k>HR2jV59B8{ko!BL_h_v*6YxD;?Kdo~v=)vb@0cS~#hLTBMR z5Jf22?6ivknAt`oeK`Vj$;W>{j3VOy9*)mOb@}DPA1COH71yIb)Bhq@l%PUu$XZDm z_yeASmkz8%oK5Hjib>(wcWBLI!Zr{*#fB-uZzi-acnoEV_k!(LsvjpqLq7jy#EAga z=sY&sS|iQgF|Pw3<|}NG%!3o<3ZqjD6{gxnLXQmo*S4pbfW;6o%H!c7Y%x&S-HNf#sHN{d_W zdtijxiBx)bvf;UVr(y_|ypD46WFW|VF){4gseN$@FUhF+FT-U*Jty*@mZ^*mT@Tjo zbBwp{ht}Heu2>lmx!sLQyLL-KN>}3DIPUU%k7)I~CgW$< z?##!g{tfj%-`1iEPCH1KEe<(QQ@g;pJ&uxzcF%H^ zx-XPJKDttEkm8Y);}oA#8F~xd3f_Ojn?Wu*pGR`4K(JN&F;jwg5?#nn(Y0+H@A?L*oNb#^WV(8R423LIrwEgr-GV7<6L-=0jKd-N)>_Kq505cTn8tJ>B$53xBKM`!^K>U20VEcNjUx=R9xPC zy&c>6a}LHP`<;>5OR@uqi_#E)vy{=jh9@z5SjK z!;zbC3%(t?q-*a)018kx69s-675$IIc;fT~G9I!)n|DEaWLbQ|UjoIhiDS+XKGSUG~~REa>4i{eqQl%9SV zT6)LGTv<9kn$g;IeC*xu$#tzulp2M%FR<;RC%nlQ&~I4f>11@zv8D||J5&EWl0_^G zD2e3*>(@(ur9dry${W1DD!nf*(|=7|=Bo$WIaS*ZAop1i~pvm|6 z&Y=aeKBa3L3M%CS#A2@^#5`v|i`evuu&48aa{y`J@|U!f`?jHWeg5xT8lbObx$n4- zw1B6SYWngbTq_@rVNs3d#e%*@^ZBPA7mpEyTr~P^=d^ZqAmr4BhBQWfy|+bHz)Y>0 z!0+*^+;4Gc2L4zbZZDaeMfYV+5{tMOQ29Z7!yQgY9`$AK)6b)=gq`bM4G$kot|ERf`^6+P6g9+6@L5Ni81wvq6}}tMJCgFzfnu7N%wY(-;WXv# zvTWIO7B=Wa&DEhwbL{ZB;9#HY0HrOwiXMI9c{cf{>H@vqEyInjX2?&fHme&V$0ZCx z_X9dPBEQR>Ut-Fg8=@Q==h?ZVZMo(2?x&xfkc9@`-|%3xX#R??337i2In>Q+%*T3| zqBO$jplbxWqOf*Gk^*Dx>Vpx0;WMQ>8`M(86g$}Rab$KuN%%k8pq9tBd3?1|Gl1Y6 zl94_nrxgLVaJ1r7EeiRScOU<0c!TX4WurF7E@Z-TS$$lu4P$ipy)PB3MkRwZ-#S7(gXLn)-fBH(h8qx8y_Zab128*7iaX0CQif7mQ#P%L&wrahA4D{1^ z&`pJ7-s4|DXf6EXVxy{Edk;MRUzVdW-`E6SJJ9WF#3jzd2crJSgQR#L+SZ7m>9nWD3*A4CHFPp;3Q07m}Ys;Z|)}9=d@^~5(OwUs_ z!T=SbZQL(=nU-OYf-BzmnA7v4$_jVtc-N)#Hs&UA??S4dC_Mq#=&pF5FPEnImE8OR zsyiH6or25kJf(`y>ltyTBVP%emG>FmB*uk_vvFy}cB7}D&f!8;1vj`94Z-z;<7VGT z#Lr7>zVoQ&taB=CnsEH2^;q9JV6m95mupgnMX~M4E~&Kgg-IN0GGO7&BW$8PDQr3o_UnnN_m*p(qc zax6lEK@3e$3|SH-PP{- zHpzADH9AK5ycmdW?o5r$X*IMm4l@@zA6Dgjd7pUK42?8;i0ACTsa4c-GRKRG+Kh||K-DzQopcr)NX}r=hTZg{T@!6{xLqMy@TM(y2zU@_4SeQ z5pX3w9hP-Nx@|VFr}kw&)~_xB|C6Izw-4O#A+T^_Na^L#RU8-p zAZRHo37j(Ih=*rr%{X==RmUpx6^C?Z2xImbIJgWc`k-gy>i$Fgnvn=d5;uwJ&03=k ztR6eL#!7oH^QbG*Nx4eZg52Xz9%%dPm>Fa)Jj65Lu=iO`-q4Rqh*_27gra!l4%j30|UOF_X*~d_h zwJjTj?TcS*g#=}$=1g+`Tu#PXYc^vE`Dv3vmaKF=$CbtI*0rp7mz=o)#{ZBv;-jpIGlNOCxaUkn|zvG&wvDian0HFihO+Ez`C1`hpH z=hX70Mf*MXwo5n*q))iDA7B!IxPKx(Gj$qN&WH7<`Zo$R?$z?5ei$m%P(gq8-hc)g zfO3zgl`if~3o-HHs;i;DGT0AM3&#fw>;|{>?Ma-9F~DE*QlzgzH`k-)^if6zKC8GA zB~&;hSLkuNoh-8?2z`NFU-xYHsy73zvXHFO?K*o?9DRj-jD3Ht_3D%UKZu(wsZ>S_ zit>a+hZ+4o!>FwQN5o@BBMx*67Vcp5zvDxgnUo*o&0dC5nseySvV@k=!v8oZXLH4- zTNerBY!HKUghk#?66IERu1135LDBB}b}*& zcD6{r(CG@g_!tyCMkpFNi(cZ6w(SS?o?8kuX0c$|R4#D~?u)>fU=XFj&r4ap^@w4q z6xMdLC(QdU#-N+nN_;$_`r6Yjn)(URaJtqKeQd6O^0>oJq$k+xaQ{A zJtpU4u)S){pRF%02&K^;mpAw;cO`+@J)49-eF?70(LW`Xy*ZSJkz^cIJa14wwj~&P zqdNRsFjsLwNUaTYf5mbRZCa4~9{G4u0;98D0)y$m7if2E_In5jLBQYt0s!yKdG(_S z(|)7}&p7*N5GMaLVnM&bt_kyU1wUTrZh}zBqg#+OEvNF6IZzg*uld>Sau|FN_DO@! zjSq$pYi~fy8zyqZFmmv+T38cdSPvQjqg#Iy}o5B zritrNb-rNaTpFobb50qlV5^>fArRGe?N6pbAb0Klf7*MisJNExjTd(f5Zps>5AMN& zL-61p+}+(J!QBJF-QC^Y-2yc3au@sTv+w_$^K{=YqaS*#(W6$^>|#~b{LSwxY+2hN zp~EcLA^fM}r_Rr#d7;}2+!Hoi-SRFJgk4$Stq14*c8V2;-5ahW3-J1pa4i*8q4er8 zVTwc+UOtJ6>gxViC#&&4D+6JYx|L1nvJkRW?(}S_yRr!;IlZ1H-Ft$^dvjp#a*<$B zkH|GDtc-$N^e7?H@n8PTmG(AMIW}zG{;}}S1a=Luv^-2d!b&~zOlOAzF(ub3@B&L+ z9M04zAcVwb4AG9<4+3;w&|i~2kdAtMmwWo=o%M6FdLHbpE<^%*W|Aqinw*EL6AZrD zl1T!w>lsjzR*GgATpa&3gs&0o_?}}2Sxmov6s@!ifA~EE*%BA|;nPKbXn zmizFAd@cN462`|V+3Q>;?`|-+xmGnc7(f(z<>UAZuNbxqlhqJgO)rqZI9i$K<{&pZ z!~&Io)6e-D4_;uPn2aa6dQOhRBMUMzbe{7?m*fR&lK|}~-fTOrS?lF^N9?g2; zc-l4_^}i=PW#;_691RUdcag={r%oe%gSB`rZ&ZAy?6qnL@j|riglQBo2=P&o#Ie`2 zZ9v&=B~_x=NH$vIZAh09kT9xrS@A4=kY`w{e6g8op5otHEVZl%@0L%}i%>r}}ie7$6dB`wLnW&RjX zDlH!0u^Avn;AE+mE65%3KKxL~?Zcq^|9invo>9W7oa7W`F5@e)6Op-H$@ z7L$te0M?l!naPtdBYqEj@}5wN;*}7PXjxaVYN0uR^ir`aCayT$t1Z=?#hSlCdM~4^ z)i3F5ICjK1!AiH$k%IyHdhcRGX-Psy(peNyWnqNPp7f;N%`;d#TRzSD;Ad0gNQ1)2 zlKm94eMG^}(TJ*qVob|d$U&vSb1}sZ71kY)bW^bl^&jyT^SrbxHtDzL z1{$SWHjE;7pLVxFFB4CuM8!LmrOG8NE5B7ef&{0WxE8)~@AH%vg2Kz6;3aL)8pbMZ zopD%0%d)7;L(_-2usWr|I`Q81Rfu`9rCsig5UkC-0Vy1mu2=j0sZ9D<<4r8b6avNq z+B-wIuM@*ChfEqB%muVFKm|PEu5sOzQPgm$*M?7D9^)9v?_UUBS} zcy2pCH1(f*OH{r4{T))l0g(aMQk^AirnQ@9>M)YL6Vs;-V*3vs*08}3_7|=Q$9{z% z7HGKl)@k)swJb?fz0Tx+A22+r|6?tv^9@ zl4VgMbdvpg{x9zO9qJSe$V6|VP#OM@1xTb|#oOH~LQG%(2czptlLx~@?^duBvzR02pAVqBG{^d ziANI_T<$Xlu!+%*r*c?N#r>W(eY{C0?PWDQpgLMAEkE%d*MKrW7v&2%n7HiJ|25Y(LbwLlUp!J6GHCdxFXE-(TUc{a~~QUAeotI4parC-C~NPOUQ?3o8)D5t!J#Pp+J}B zNp}AUz>HWX4C~+VTd~0J{B+Q| z!PV*e`cepXwQb4W_33rp=6fCMu1I0cU5w(Q6hGmOg-VP#GEuvXj$cR&y1Zwl!;dPY zXguB1sf-5-8%-aeJ%x1zanb zo>%B6_}ni&abKoUUfE(j+e9v;GZ7WDA6+1gt)&*f1~@2806Ehho{Q%1lV1T~AG-ss zhD69NIperu6Y?eQ3sbR5YgrzoNA|u&KEMab^AymBz;**1sJ}Je$l3CYLZ;=4M^6?` zg5IEGpmk}YHmQX=Ew{@76EfaYR37#&`&PQGq2TgQ(r=Ao8%c~D?*1eFsYo+1> z@XQLLvQu2XYbI0V49%hmTt1dQo0@sDuc$m#9&+`1~L%taWs& zZg#^gL0bo~CdZ1+Wp70CC-WQUui|IggSGhDzEik?TmlV);fkKCfcwB8_El{*9(2Tr zBiXOr(kVQu434E%EKzNRUmXYi#QPHR%0a~{ucyB)<Ub@X>> zzO}kV=AmEceWWbe7%%lioUuyT?r0s#Z5)X@gOJF&5ep11B00({dL4*f!g6riS7OAp z9^hT#9zb212YKH~wEisC(2Ckxyg;+yDewa=*Xu0d_kxTYt_ZMn2R#h(ol)Y#6u__dYE_voJM zp;2ST=vn3#T&(yCbN%It@(q;UpyF(JKbYu9F4`Mi>;e`4(UDRe zUe)0>={8H+*52OyIY zayzclFeTP)v$1=lIwGT~Y-ASOdS`4;>6Kg9ky-%dgG<+LZ(BVCwuMGt&!^=C6iG5@ zcmT1u&)5ZNwJhP%*C>KT)3cZ#4%cRPR~yVRr?Vz#%D**#e5*H;Gtv1=ZtQ~LK8rU1 zwl697qpwKT!`}DGFwLgZzkc}J02`Hpc|49h=G@?^Sz1-s3zns`@X$^(BKb%UlZqX9 zqY#u~X)b$S5z%nWcLkE;@z{+6amdL&+}V#8qyI3v$2r5t)aSDn)(a5KP;APGO>k3KqVoDbbS991+rVE~ z*9GwOil(ggEEJ2}7Y$-2>CGZusI^#2a=USn+Q|jH3<4xn%F?(3=J6N=&_2qqdkO@< zr_&_EU}`AUX;*kM1gg_`@ql!4|G}g<@eh-kTwcxSmW@D(Ft&6U#p%e{7G1wXB^^fX zmJg<4T)cl1OQQyX)=s{UyOvCwsemBu6@JAm767Cxs1xe-_nn_M%D`kR z-|m+~>&kucw;}ly=O&;36}st9Su>SX!U);zc!K-4yWg3DYQJw~Pf`_|e|HrZzx}J( z>`}tSSV?R;r1HN|J1IJbM^%2L9-k+w5AE$<|&7_*; z|2Ds%v&q)Z*yUG1f8V>|bokRdt9a*~i2%fAS=W+Qr3S3m0X8O~fEVwO{`)17nn~G( zn+?%yqZYVN7T>*(mJ3qPq+_|+l#5l=7e2qgJzXuxv5A-Aus{;~!69p;QSx2y2eXT) zj?XL5>2K--)v+yl=ZKCd&kqgE-FezS&2MvgLGrq)ZIrAK-Hy>w9!MX}$wQ;$j&mE@T8-xa=Mg=*9p`+zPt$ z%bs69I5wZCP_H4;bPZf@7mvmTHRxTjSue_FDC3Dm;aRt=S2?Bz+5+7>h#O5dH_ZSEE$clY^S!D9<|(Uo8*0Ase~vl;B8;#0kwbcI#5YQf$V{%kufT; zf-}IQ&4;#+4vM(0wU`xJafxBHTKWFy(nWF0OLCSx{dnouyOqd8;`$1h9JJjKHb0ac zCM5nMR%TzW2Qg{7z9VUII$g1BARVQe2aHgDJZ-#`GPFkAw)`Xmt)DLS+pw;?bathv zu=W*6`}%faJjK(U3F(qIIsRe)@~sVk)|B!#hxKdZB1^oBa`Jq zVh;%$$F&o=EBbY^U$cy6wDc{8;B4WoEvkytX6=iSW>Uvo|pid5I=(Hif80K!pRVJS>({ zWJuz{@@z8@g*@8#fg|ISM4Cr}WLGZTa9rszKHnanE5FTIOiLz!Gdpzx7aL5&8=%8u zFWLb`**l^Wswwk6%Zp&eUK!D%3)35e(|(1vjq|mDL7I?g=f>Qt;TX{f3Cza9UqdD& z7MR42jx-u~3RQ&wKSIE!ycZR)CN$4Nxqwn!piUd{Ok>LkcqtjDdJ1OcKUMnRRA{MN zix)7ffL3=^Dd}&i`d-Pl8&&#;p7}~7z!3FaI)_jW&VG<9#as+}a(ayv=c_F$FFP*R zfzWQ$(D!}A0U&tQZLt{~CKAo3JQSzkz=ep-KtemBcHelr=8zL~wViff<2Os!Ku zM_YsIYC@`7Mi6uh)U`}4*n%*eKR9H}4hb3!^BcAFAJM58sg`ORG^kszRCE+2;P=&< zsc@Wqx>V&cwpm7_o(GbvrY?htjL`!usCDqc#K2mfR^wJay3=e`F-_o9-V_>iq+5%i zu@8W|B{-17c2MF^?=z-?>8Jt>F{^9k^WxEa#2`P3_GIgd^TLYGI%I0ad?g{|WX(}W zh+T}->yD6_y*_Hx?3z>+y7VImLmb0ZXy4Rno_e=)GSEOaLzWkqs_0*q2=di$mI$0Y zgTE0>oRRC&6?s`M$D(sq8tn(^#{!LjsB;Y~*FMK*xU+{fUw6_9V4>VA8=jiSEdzB8 zrkh*&IbNZw8fZ%x$6x(vlw8EhI-geT$N7SskB@!k?ABv_myPAKUg)jYcrRXT^2{h^ zoF}4<1~%4oBm}4BDmat<+}e->=pSURS|7>Jp}IZYZ^dct%|&FZk{K+&of9Fk`wp?= zkiYk}5%Ki-2>$SmO+^S|GBZWlU~_NIxkK(V1)stls+7v%hRGXK)Nz)OtTJ+s3B#iO z3~V=Y5LY?`bvJ*(-44dCn{~uBPqQfys0DV;HRlZfSS~0G*sF2=TSL7&dR|ngJGmgN zm2Vq6*HbK*skt!fK=z{EBEwa&zWHGvk-B*?A{9rz#!WXMHa5 z8$oaQ{X-H^L6q~x{_gGf;oXK>sz@gFs}u@#y&lZ=EF}H_n!hYLfn-ckEDHIDijxS5 z$YiBtVc1KAVtwDZ)Qd#FZp}tJ(J8cJaPD#(agi0gjzX8xd8r?6Ikz9$mt6vVQ4Q4( zV-jeTHLng`gGD|LTDLCB4pm#flICTJ7o^154s0lmunq-DIV9F|`(xDhp5(N`x&w;zm{x^0Q@l4c6-ce%_&z^Jk z3heaQ;yH($ehG-tb{YDsoxZ2_?QzfeT>J0x&d^>{n0?>Z#ER) z@Cggmf*Yx?v_O$e^WaId#%ltfJG&nZ^;q}wVE9MxvSkuP0W8`CR@g(Xq0$SRAx1j;{bqA|ck>3A^*&X~!NP5ZEEa`LJd z1st!D@E-)O^|L=@?toiHYeRmZS5yA%578y0RZ@5pS_7AlLYm4N9)EnQwVwA-lKbr> z(9KS;>SdTQxizM$Et{LHMWj)<{QbioDsPO$Q}JrbQ(Dk5su4y#;5C3>oRTC@zjsLw z@O)&qECgfD=8b?#k%M#bLAJ5)8%T(7!J?H#DuppRd(5E_&jb7T6%cB9--RYwjoDaI z09CfD>Ow9B7Dr2fI*yl88^o+Cqe8&fHSN1D)T5yczDFcbfowYaX}4qd5o;I1zM>%W zZ&-=gkD*gU2*Q9@hRUybaO>5jQ6F$}orYQHSr|QRoMtd%JGX)zO8M4?$J6FrX$TZM z;dSjF+;A3adP<FsMtf4E&Ei5?vbU%a^W|Fz>vu zKnS#5WCU;~2ggo6aRT+lX5N*5(%9{kdF5Y&>XMW#cQpw{BC$=h9twuy$ZAtEK31VF z0(=Oaz6~Bjwo4Wat{HsDqLjNBQ<|6a7HTEzEC<&K$L#_atHwL~9n7ojLehOxJpEG+ zGC269t`p7&wAVsz=fAxGNUzhwR6n;hF`|ClS?SkXt4IkGgGuiYcs-7fw!qpE*8cva zZlZ>QrB{rY?)RYo0cHKO7O*YXhH~$85eP@U&$;kBc5xeg2a{Bxt6SH!L6N2^@Wq~9 zTcO|ApiqXiJ@rl(OJx>v(%O9P-JhCaJt~5r?hgFm4nebJG>ukH-GEs#3v3z$ELbdU ztxG^@eby($M(_>>YgdfOe_pO&Jd4F6OHl#XKpAZyBBF5mEc6@JLLcc>s6fQC5WFbC z232}?S2cX(CgEJLdgHEA`k%^&M0UVPdC!dR~8e4b%nef=UClY86v+E zxmI;k_`TEkYMM*|S9fX0T4~rz(sHLIaM^DbB|$ z6fk1^k59B+3{)uV&DYkHE<#X7aC8~lT2O_P+U-u}xA>=O4i7JnnIc}gvqjsft1CaZ z?_SE9pYh<2Xb08gAw0W08A>*VGWcJg%*BbE7}fhTe_lXkV-j}-2|*Sb{8o11d9|{# zT1e_{2FBi==awz}1*F`21ylf*CRDh2tSZAi$nOey z{i)_^kl~&1d59qdhIADFvjfaI=_lL>T}p_AEJ3x|{vwzaq+N`6<~ok$a`hXU_dGL@!E(zmP zkHJoKd_8~=JF(YaI&;@|R#*VHXLo7|8*NBq06s2qH_UV@V zDcT4aJH@o_?hJo1fwR!5lpyY9vx7YfP{UJEGch( zs=sOQ7q*4cLrWaJQ+lO6G_Ek8AtVV+gJ$bSNGLrWw_Y%sCHfE&ViO?cMb4h&IfrQ;qDIurC3MRB1q`RwKhVj`gUy13fz&mt1j zsgbNY?NESd2vGAZU9;0%z818IHg^rdF(G0%>JeD*?vJHkCYL%vEL{W{kd#2c zFtys4amcJ$ShvnV*NC_10<}+^?e=BZ9f7L(!VKbT$}A<@i^dEn1g?IE+k}BJmuS-O z6o^+JKDF!{ZYd%;46HS6o0MejfTte3!kJRTdQP!_oKN z7=nRP?wkB%VpXE$l}Ng78r*DS;oIXd8V$#`S6s1D(H2~& z9CF)k8Kw=?hJo7Kj#v?Fc_9Nbi0gz)p|l!HK1zws-5yTemqM$po)d%=r0W3C&mn6? ze>?2N&gge{sK9Ma4soG_gKNxJfY4GdL^^ei z42_>GA{*47&E2XQ?Y480bF_5Dqt!ngy_YN8J?X;++1636KsjgUjhn@ ztVAZC5r5)QptAH&MOvsmH-A-QEQJY^CGsF#JtO=o|)BusO0SCTn z#lnD%e!?Hsm~HIKXTGbU+2S~A10{3Sge-7inP_0VL4oRI)@n?o4Qjo~KSByoZL4n< z0MOT}lzf5CKj`ZRoM5m=*PVh#+t4esII{E#8QAo=;xE~P#G_9?t5ql$3&Hqz;2aCS zu?h1Czpv*y>sQGV(|AgRgUWaD=M(bP@F}X?qA$b30uXp|cpyvqS+WZ|1Eb65qN`_ zPt=I%@BXKAEkXc?+>%i^2>2gV`tPoheUn5X4*uq~|NID2sJ2%F zy}q~?^w{1UH&$FACnU3#&zI{I&}+Qp;B#0(4buqS-I5W*ljc(C4BP>#Nolx6cxu$} z_w@Ow0uLYZDrjObqW?Y#K*2S#y13R7{+Dbp`RKnXrkI!TZ^ic4wy37tW4ZL^t31U5 zIsBHZ$iD*f1@5g|1aV|A=YVl^eyzDRvV#Xd@%0g3%**X+8WnYit}XyDdro#?WC}Nn z=_76v8FixAZNWg)iUreyik0(1dOlH{vYGq_dAscmm3!C0#QO?>Rqq?}d z^Zc&B@EWsOyf=UJu8q^@Td`iJtCUwlpAK)W0{>FQb~{2aK*AFOxcj@p#>I@~0P(gd zAbkU&@>{@}8~G6Tms0?)iCma_ZWKDLf2mE+nu_Iiq~Y#uhM<(o$Z2sUPIP9|DKWn5 zAN9_Ez(7Z*z&+pF^o3pYh>9bYE7neV z`v!SFdUML0>R3(0ouMj!5PiHkDYzkA`_*zU^%bz0myi*ead+f%yGlAMQK)XaE4`kg z&D&D-assUh1ODsOpiOJ=KjNu}fPNDlCIm1-&oZ0+^UlR5CgcBrPl`NZ+HsOJ3PE5o zy@17(u08c}gv=Xr9FQ2%G8`>Zqqt~3FN0=^9L33BCmid@cQB6hbjF_&FtzEw)fPa) zqTsDj%4A5UuN_dO}p`(ftArV zbr_nyvw(=|kGrnh(G1{U2<;H;b7DTHm1EJJI;+;c z3nDwHS<;v;n4|cKdGR~Z@I+DgaM=1f(OEs^f^yA9$?uQy0zMo^c!mqo)#?c&*Y>Y3 z8rK*v^7F@27JJ#JrH(_?PuQ%+RRrFYmIOOp5=2K^34rMEP#&;u zBJ)QmwBc25!rXOzQs8f{ko{We`G|5@Qf)j+H-4Z5AP36Z-mv$-(zvRJJzKfi;9r3R zK)lJ5u5Knk+GcXyO_~P$>j30}2O8m+*w^WY%>FknQ$7s|M!~mq>E!a6PpQkM*QlS& z@@@S+S$sXn9mC(}AIxRHSDE2PT-0f%xKP&bJI%p`e7VP%vp0$tiVuVEX8zv-&?ph0 zydpoP79U2sKw~LFL~M;JBe7iboB$SvWmKakx5jjO0-zmKs$&FzDlqu9-ua{MS4>p` zMwMN8Kc6PJfl4wGeNITf5Rw8)co7tE<_-ESbRkUaIAQ1<1(6sWt{#lDFHAy_h4(Gn z=xAsXPzET1?IdCysL;8;bx!6iK*u8vw+!l0_X2IJrcKRdI?A_gs@K{wpEuzXgm^+p z)xTqbHU;p_#E=Bm8<(@-T!Hj>^}UUjEd>h2er%W5HvfZdm9m?T>oqd##YK1;B3>U9 zoeh2S-VyCc-N|zJwR&8Y|8{qxzuhzJazAzy=la#@>S~rqsrqGh@oAY>vz9b$*s-7M zZL$!g_tC>n-i@V9Fv`gapI*DuQcVfbL|^_moxRsdB;IZ^%=MqfyogN1ARQ~4-_4E} zP^^XnVE%Sq+YOhG#2Sj$NT%hoS{c?Yd zeIJ>7zC@?Jv$RbYi7Cjs;oNvXIzzxJf-A=u+2r8q@Wff`#eIv*<+9XyWI!Qzv=f%t zLhG9q+ll2X8WVn8g)IR^w5}I{!OXsfl}Xy#w5HP`oa!FyIB#kLG6!p)CFSE0;-XGYhe(rm_1!m#5#)yZ1tC2t-`#KL|`yZQ&ry?4LION>1~)8@VCW!5Vg6 zdkPNc%iO*Q&ls0>WJr+M^!}OoTl!G$yHktlQcb1Q;NxAu=Ugu3Hf~A?G7^YKTJT1q zc*XI10kRP%(8_)7W9~|mBO-p*{SQTcg*pYBZ?zVBc6O3W)+bA~%y->e*lYD5E(){A zspVF8Rp*Rd8{_ii$YQ16`jzg%NJP3pD7b1PCs4g`^#yorMH;&x_5GI`n@)o13O&fg z_w|K4>h^I6ae>ovpv_bDM!VY7$TYS~lQe%Qo*7f;-?F3~PL>vF&dQs!R-_nlGg$U1!ihftSxVb^v-z@JOeGLQ0*g>sEKb-Z zQPOTivsgc&c@R?TK$?MS9Md~A&9T!|*7=)7j=FIg3cC;u_T7rJ4nKWwkK$mWV?^*Tl`QPCO?{(=D#mYRq(*%U)TGOG^#d8C+gJjdrlGo9k>| zN~+ybqte|P_N%%jPDYMq&4-_W?WvysgYjO(;V--0p|U$5t+2%0#H$eh*U%C%_VuSs z;5D?1=AKH|!wOPHjF5fs*>FTM2A!(53-NI%wqHEYj@AAY0n+cxtf2cqd~aWiRlON! zDW+BKN3Rmo$Vvt+?quQx!ObXYq$Q>CfkYiE)?lJ<6b9+t`l($@(hnG^nYt>ASer_q z9bFy=y1xe#F2e4N{dLsuKPreX7xU385p-d)tK79RJoqak4C2ct5f=b;U09rj*rl?n zJz5Z$&<2h>l7DEs+d{Oznyh^?y$wb8E1m?EK0+aUU~M1hce<&wj8xh4d=$@&hqwM< zQ|kAeDB1F(gDnmk%K3xw7&@jHRN_0vy#EvL)~rQCiSPYo*~lQ5T52!{uOvslY)`*-EZlV|b_o3FGxmwwvSq^r?1C?_!FS zQCa&JZ0iTh^hw!@RGl+Z5to~~$3}86_xJXiE)~3ftZ(&WbN1Rk2StxcF}CZMwHcQr zyFH57*D0r{m+D)%ll$`WI38^vLHlL5X8XJYMM9mr-nPb%`5;E|jKKz0wcdHKdq4a( zH44K=D9BhEjMQ`L(a!}_k!8xtfg9i9+E3fmJh17;N12+CCN_`e?n0(SF4-yDf${=!kN=R4KS=-t8 z*zIWqbLt8E*h}Mq2_^H>9c@e6UC5=?d1%sa@0a`CU*hG&%Onrr#)7YzI9oV=KaX(r z;@Rh;gg^-B|jzx~oCRG1BdB^oBH;Uy5k7XiT1v&*y1 zQVY>W^_&*<)R5X6IzDBGE`j$Pe}9-?S5P3<+T_96VqLGS;i4%ZnvE8b3eVOENJB8d z^YAE_s4z6DUXi{M=-|;sHKJF4Qg7mq*J^R>iq} zA9aflXcsY>nq7pntDI3k#Y3eP(+1#iD0P;V<)I9C?v5jHJ!`qz)MG`C5ODpxhsLQL>%8jRuKI<#mv4ZM29Ca< z13w^7)t_OmXOipP?2t@*F8Qa;=d3?tajZ8y)9KaP$zzgO%bD!#PmC}^6C%ZR7E;(( z5{Xb@2=Qdw^ce5f<^+O)2p(|B^?fH}d{MM}kn?7cKZi>a`spHh)7BvZ)f1LV{YGm( zS6`%m@;GDNOMjm>63w0$7Mp-T_mRI#bfNlmn0l*>oepgicAO*6~k@+|JS^pYkLzoT6Nkysa01p3U|tn~at3;p*Eyjg*&jYzw4s z1l@M51Tv_6XqbvQeW|!?SgBM8D-*ikE?DR|yq~i)4vD0O12H=JcSVK}`{jf>cNj61 z+fY!c-9!=cq%nH8dwH%LR`J6|C}>)$F*EHQi72e0ZHmO%5+sO|ZHK^mLFAgv3a^@p z+p$=ueaJ+8X|Dmj*CfNO_8;e&zaqBrbzQ-;IYfdMn6qIl(QfcdC>!f+%I8Kgw=eqi zujvvN&-cu91hTZ(y^;?H>Q3)+dg&q9y%Y4l*GJ&A!7?uv-N(>RtUFJ67$dzIo*BLbX=BL$k60R`@bK`kX1!!Q zq6$q{TPe4q7QC#{F3c_}IXueo$G~w9ZoU2A!2ZXkxq7mGC~wCT2eI_DB9tUENp!a zSjOX_h-|R~4&_^&fXfoYVQuhW>I-S@5HvT_TAG3fuYqeGLo4pC>pV^qdF_dJH(3QO z2`+g*UFwvLTKpM0`l{yL`mmI06&T50wT)^iD6i7WYwX;tXTkdM7bZWxXG!zBD#{E< z&3tx?yVORbtrG?pys_gIjDN3x^!krO0+@SHI z2s!$qEso*EOs%YhkOIdOtxsgHJ;Z*q7Hi?${8nks$-8yjhy_HE+s*}u+S8SAAwF=& z{#VamAky|{Xn(?sgjs(7o8fig?GuOe# z4UpV!h7ie~%zf<0q9;sKQmBHe_7OysC!Vo_xQ=uD@?c=P${pYyPhH)k?RU!iLg3W} z^(U&?^rzy@lPmEZPJV`-qxn)^CF?ISEw;XrZ}(!Mb>h_i>c(+eb;nohVUgg$eo`8W z@I(6RUcxIBEAqeOQsWvTlWm?hHoH+7v^+=`e+iBV+X?Jp-l&sdxNkgyvIM>D`4kr^ zE@bc+JD<4vUI_v{`}l4fiMNqN1q3DvAYZeo?-^k zAH!+#-|xh(aQWpByPbKNgxp)&t9PcjEz^f8SM0dO#;h6?cJB;MpOl<4C1P8kzj~rl zH&UE$d95g)w%Gs5CWO0AQ@#td(R+5RVVg-NytVxB+@sNls}K8Ba=#g@%4*OTepsnrVgY#S1&@c1D~&sxarbjX6Pwd4rO(-b zcc}^57#Xmgknr5ZD0%KMEa)h#(GQL@1rZ9(9M=U*cqeM|t`I7k=<9{VOWb-G7qJ%0 zS{MtC^pZ8_NNIk#SU{2jEz1PjWRmX%mlja~OIR|SEVM%1`r1ooQ&`DVlsl&iK8m4O zXXLPDT}^pll1=_?;1ALIE+GweSvCiAu2h5P=R0)i@nZtVp6d@~;Yw<&Nk)dXUDo}$ zbR4J5witKQ?wOA}S0e%ADbMD&G7D5-7OKgo+Qt;L%UK7yx<+K)`p9(_mA<>a=h4{{O_nD0(C3214cpqv{V)OridDlG1oC<-DR7*q z_6EbBqH^~XAmoEEN(`-58WzjEh?FgY%aA0(w89>(HsvsjLmGzjJzp#=us|se^Q^PU zwnNpa4D|8iar_IMYZy2`ZP(p9dH-Ijy#p_$I+kihuVjG2Oc0f~SXiZqt+1D9brWqK zF>Kz!Ej|F8 zFS*Yj&0F=$p0aDz=pxUe2EPnH*?&@1#w%Rn3M$XO|I^Ce6{mb+^FJCwn({m%Hr4K!EJx+g{ zuq72li3&oA=vu#v$*$lyEVe+W5{FoAc4j^<_qKY~JADMNTT)-Z)_KYeZ#Z6@&Lr05=s!@`R-0oNn78BpCEr#i1IH`adrJ|6lq44fOxVD4z*5a8muYfXWsF1O6mLWkt$9>-zma DuSZ%! literal 0 HcmV?d00001 diff --git a/doc/usage.md b/doc/usage.md index 347ca35..aa7572f 100644 --- a/doc/usage.md +++ b/doc/usage.md @@ -371,33 +371,17 @@ $ torchrun --nnodes=1 --nproc_per_node=8 train.py --config ./configs/7B_sft.py - ### 长文本生成 -在推理阶段,您可以在模型配置中通过设置 `use_dynamic_ntk_rope=True` 开启 RoPE 的 Dynamic NTK 选项,从而使得模型适应长文本输入输出,达到 16K 的外推效果: -```python #21 -model_type = "INTERNLM" # 模型类型,默认值为 "INTERNLM",对应模型结构初始化接口函数 -NUM_ATTENTION_HEAD = 32 -VOCAB_SIZE = 103168 -HIDDEN_SIZE = 4096 -NUM_LAYER = 32 -MLP_RATIO = 8 / 3 -model = dict( - checkpoint=False, # 进行重计算的模型层数比例,可选值为 True/False/[0-1] - num_attention_heads=NUM_ATTENTION_HEAD, - embed_split_hidden=True, - vocab_size=VOCAB_SIZE, - embed_grad_scale=1, - parallel_output=True, - hidden_size=HIDDEN_SIZE, - num_layers=NUM_LAYER, - mlp_ratio=MLP_RATIO, - apply_post_layer_norm=False, - dtype="torch.bfloat16", - norm_type="rmsnorm", - layer_norm_epsilon=1e-5, - use_dynamic_ntk_rope=True -) -``` +在推理阶段,我们可以使用 Dynamic NTK RoPE 来代替原始的 RoPE,从而使得模型能够适应长文本的输入输出,达到 16K 的外推效果。 +目前 InternLM 支持在 huggingface 格式和 InternLM 本身格式的模型中使用 Dynamic NTK RoPE。 + +1. 对于 huggingface 格式的模型,dynamic ntk rope 目前是被默认使用的。如果用户想要关闭该行为,请将 `config.json` 中的 `rotary.type` 修改为 `origin`; +2. 对于 InternLM 本身格式的模型,在推理时,通过在初始化模型的配置字典中添加`use_dynamic_ntk_rope=True`来开启这一行为。 + +用户可以直接通过 web_demo 来直观地对比查看 Dynamic NTK RoPE 是如何生效的。例如文件[长文本示例](./aux_%20materials/long_text_example.txt)中存放着一个token长度超过2200的文本,如果不使用 Dynamic NTK, +模型是完全无法回答该文本对应的问题。而使用 Dynamic NTK RoPE 后 InternLM Chat 7B v1.1 模型的回答如下所示: + +![dynamic_ntk_answer](./imgs/dynamic_ntk_answer.png) 关于 Dyanmic NTK 的原理,详细请参考 - 1. https://www.reddit.com/r/LocalLLaMA/comments/14mrgpr/dynamically_scaled_rope_further_increases 2. https://kexue.fm/archives/9675