BEGIN:VCALENDAR PRODID:-//Microsoft Corporation//Outlook 16.0 MIMEDIR//EN VERSION:2.0 METHOD:REQUEST X-MS-OLK-FORCEINSPECTOROPEN:TRUE BEGIN:VTIMEZONE TZID:W. Europe Standard Time BEGIN:STANDARD DTSTART:16011028T030000 RRULE:FREQ=YEARLY;BYDAY=-1SU;BYMONTH=10 TZOFFSETFROM:+0200 TZOFFSETTO:+0100 END:STANDARD BEGIN:DAYLIGHT DTSTART:16010325T020000 RRULE:FREQ=YEARLY;BYDAY=-1SU;BYMONTH=3 TZOFFSETFROM:+0100 TZOFFSETTO:+0200 END:DAYLIGHT END:VTIMEZONE BEGIN:VEVENT ATTENDEE;CN="Jörn Tebbe";RSVP=TRUE:mailto:joern.tebbe@th-owl.de CLASS:PUBLIC CREATED:20210510T164456Z DESCRIPTION:The AICommunityOWL is a private\, independent network of AI ent husiasts.\nIt was founded in 2020 by employees of Fraunhofer IOSB-INA\, th e OWL\nUniversity of Applied Sciences (TH OWL)\, the Centrum Industrial IT \n(CIIT) and Phoenix Contact. Together\, they believe in digital progress\ nthrough the use of machine learning. They want to create sustainable\nsol utions for the challenges of the future: industry\, mobility\, smart\nbuil dings and smart cities - and above all\, for people!\nThe Machine Learning Reading Group (MLRG) of the AICommunityOWL has the\ngoal to get a better understanding of current trends in machine learning\non a technical level. The target audience are researchers and\npractitioners in the field of ma chine learning. We read and discuss\ncurrent papers with a high media impa ct or prominent positioning (at\nleast orals) of the leading conferences\, e.g. NeurIPS\, ICML\, ICLR\,\nAISTATS\, UAI\, COLT\, KDD\, AAAI\, CVPR\, ACL\, or IJCAI. Attendees are\nexpected to have read (or skimmed) the pape rs that are going to be\npresented so as not to be thrown off by the notat ion or problem\nstatement and to be able to participate in informed discus sions related\nto the paper. Suggestions for future papers are encouraged\ , as are\nvolunteer presenters.\nWe hold our first online meeting (after a yearlong hiatus) on Tuesday\,\nJune 1st\, at 16:00 under the following li nk:\nhttps://th-owl.webex.com/th-owl/j.php?MTID=mc7956692614f4c1b692161ea5 d52\n76fa\n \nTitle: \nNyströmformer: A Nyström-based Algorithm for Appr oximating\nSelf-Attention\nAbstract:\nSince their introduction in 2017\, T ransformer have shown\, that it does\nnot require LSTMs or convolutional n etworks\, but only a self attention\nmechanism in order to achieve state-o f-the-art performance in many\nNatural Language Processing (NLP) tasks. Re cent examples are BERT and\nGPT-3. The abandonment of recurrence and convo lution speeds up training\nremarkably. Unfortunately\, this only holds for short sequences\, since\nthe runtime and memory consumption of the self a ttention mechanism grows\nquadratically in terms of sequence length. We wa nt to discuss several\nmethods which have been proposed as remedies for su ch issues. One of\nthese methods is the so called Nyström method which us es a matrix\napproximation technique.\n \nLinks:\nAttention is all you nee d: https://arxiv.org/pdf/1706.03762.pdf \nNyströmformer: A Nyström-based Algorithm for Approximating\nSelf-Attention: https://arxiv.org/pdf/2102.0 3902.pdf \n \nSpeakers: Jörn Tebbe (TH OWL)\n \nFor questions or suggest ions of topics\, feel free to contact\nmarkus.lange-hegermann@th-owl.de\n< mailto:markus.lange-hegermann@th-owl.de> \n \n \n-- Den nachstehenden Te xt weder löschen noch ändern. -- \n \nTreten Sie Ihrem Webex-Meeting zum gegebenen Zeitpunkt hier bei. \n \n \nMeeting beitreten\n \n \nWeit ere Methoden zum Beitreten: \n \nÜber den Meeting-Link beitreten \nhttps: //th-owl.webex.com/th-owl/j.php?MTID=mc7956692614f4c1b692161ea5d5\n276fa\n \n \n \nMit Meeting-Kennnummer beitreten \nMeeting-Kennnummer (Zug riffscode): 163 595 4572\nMeeting-Passwort: 67C7s26XiVN \n \nHier tippen\ , um mit Mobilgerät beizutreten (nur für Teilnehmer) \n+49-619-6781-973 6\,\,1635954572##\n Germany Toll \n+49-619-6781-9736\,\,1635954572##\n Germany Toll \n\nÜber Telefon beitreten \n+49-619-6781-9736 Germany Toll \n+49-619-6781-9736 Germany Toll \nGlo bale Einwahlnummern\n \n \nMit Videosystem\, Anwendung oder Skype for Business teilnehmen\nWählen Sie 1635954572@webex.com \nSie können auch 62.109.219.4 wählen und Ihre Meeting-Nummer eingeben. \n \nWenn Sie ein Gastgeber sind\, klicken Sie hi er\n \, um Gastgeberinformationen anzuzeigen.\n \n \nBrauchen Sie Hi lfe? Gehen Sie zu https://help.webex.com\n \n \n \n \n DTEND;TZID="W. Europe Standard Time":20210601T180000 DTSTAMP:20210510T164440Z DTSTART;TZID="W. Europe Standard Time":20210601T160000 LAST-MODIFIED:20210510T164456Z LOCATION:https://th-owl.webex.com/th-owl/j.php?MTID=mc7956692614f4c1b692161 ea5d5276fa ORGANIZER;CN="Markus Lange-Hegermann":mailto:markus.lange-hegermann@th-owl. de PRIORITY:5 SEQUENCE:2 SUMMARY;LANGUAGE=en-us:MLRG TRANSP:OPAQUE UID:040000008200E00074C5B7101A82E00800000000A0B86ABCCA45D701000000000000000 010000000438952A6769EAE41B877BAA9621B9A18 X-ALT-DESC;FMTTYPE=text/html:

The AICommunityOWL is a private\, independent network of AI enthusiasts. It was founded in 2020 by employees of Fraunhofer IOSB-INA\, the OWL University of Applied Sci ences (TH OWL)\, the Centrum Industrial IT (CIIT) and Phoenix Contact. Tog ether\, they believe in digital progress through the use of machine learni ng. They want to create sustainable solutions for the challenges of the fu ture: industry\, mobility\, smart buildings and smart cities - and above a ll\, for people!

The Machine Learning Reading Group (MLRG) of the AICommunityOWL has the goal to get a bett er understanding of current trends in machine learning on a technical leve l. The target audience are researchers and practitioners in the field of m achine learning. We read and discuss current papers with a high media impa ct or prominent positioning (at least orals) of the leading conferences\, e.g. NeurIPS\, ICML\, ICLR\, AISTATS\, UAI\, COL T\, KDD\, AAAI\, CVPR\, ACL\, or IJCAI. Attendees are expected to have rea d (or skimmed) the papers that are going to be presented so as not to be t hrown off by the notation or problem statement and to be able to participa te in informed discussions related to the paper. Suggestions for future pa pers are encouraged\, as are volunteer presenters.

We hold our first online meeting (after a yearlong hiatus) on Tuesday\, June 1st\, at 16:00 under the following link: https://th-owl.webex.com/ th-owl/j.php?MTID=mc7956692614f4c1b692161ea5d5276fa< /p>

 \;

Title:

Nyströmform er: A Nyström-based Algorithm for Approximating Self-Atten tion

Abstract:

Since their introduction in 2 017\, Transformer have shown\, that it does not require LSTMs or convoluti onal networks\, but only a self attention mechan ism in order to achieve state-of-the-art performance in many Natural Langu age Processing (NLP) tasks. Recent examples are BERT and GPT-3. The abando nment of recurrence and convolution speeds up training remarkably. Unfortu nately\, this only holds for short sequences\, since the runtime and memor y consumption of the self attention mechanism gr ows quadratically in terms of sequence length. W e want to discuss several methods which have been proposed as remedies for such issues. One of these methods is the so called Nys tröm method which uses a matrix approximation technique.

 \;

Links:

Attention is all you ne ed: https://arxiv.org/pdf/1 706.03762.pdf

Nyströmformer : A Nyström-based Algorithm for Approximating Self-Attention: https://arxiv.org/pdf/2102.039 02.pdf 

< p class=MsoNormal>&n bsp\;

Speakers: Jörn Tebbe (TH OWL)

 \;

For questions or suggestions of topics\, feel free to contact < a href="mailto:markus.lange-hegermann@th-owl.de">markus.lange-hegermann@th -owl.de  \;

 \;

 \;

Wenn Sie ein Gastgeber sind\, klicken Sie hier\, um Gastgeberinformationen anzuzeigen.

< /tr>

-- Den nachstehenden Text weder löschen noch ändern. --

 \;< /span>

 \;< /o:p>

Treten Sie Ihrem Webex-Meeting zum gegebenen Zeitpunkt hier bei.

 \;< /span>

Meeting beitreten

 \;< /span>

Weitere Methoden zum Beitreten:

&n bsp\;

Über den Meeting-Link beitreten

< a href="https://th-owl.webex.com/th-owl/j.php?MTID=mc7956692614f4c1b692161 ea5d5276fa">https://th-owl.webex.com/th-owl/j.php?MTID=mc7956692614 f4c1b692161ea5d5276fa

< p class=MsoNormal style='line-height:15.0pt\;mso-element:frame\;mso-elemen t-frame-hspace:2.25pt\;mso-element-wrap:around\;mso-element-anchor-vertica l:paragraph\;mso-element-anchor-horizontal:column\;mso-height-rule:exactly '> \;

 \;

< /tr>

Mit Meeting-Kennnummer beitreten

Mee ting-Kennnummer (Zugriffscode): 163 595 4572

< /td>

Meeting-Passwort: 67C7s26XiVN \;

< /td>

 \;

Hier tippen\, um mit Mobilgerät beizutre ten (nur für Teilnehmer) \;
+49-619-6781-9736\,\,1635954572## \;Germany Toll \;
+49-619-6781-9736\,\,163 5954572## \;Germany Toll
&n bsp\;

Über Te lefon beitreten \;
+49-619-6781-9736 \;Germany Toll \;
+ 49-619-6781-9736 \;Germany Toll
 \;
< /span>< span style='mso-bookmark:WBX53126'>Globale Einwahlnummern
 \;
 \;
Mit Videosystem\, Anwendung oder Skype for Business teilnehmen
Wählen Sie 1635954572@webex.com \;
Sie können auch 62.109.219.4 wählen und Ihre Me eting-Nummer eingeben.

 \;

 \;

 \;

Brauchen Sie Hilfe? Gehen Sie zu https://help.webex.com< /span>

 \;< /span>

 \;

 \;

X-MICROSOFT-CDO-BUSYSTATUS:BUSY X-MICROSOFT-CDO-IMPORTANCE:1 X-MICROSOFT-DISALLOW-COUNTER:FALSE X-MS-OLK-APPTLASTSEQUENCE:2 X-MS-OLK-APPTSEQTIME:20210510T164440Z X-MS-OLK-AUTOFILLLOCATION:FALSE X-MS-OLK-CONFTYPE:0 BEGIN:VALARM TRIGGER:-PT15M ACTION:DISPLAY DESCRIPTION:Reminder END:VALARM END:VEVENT END:VCALENDAR