Page 14 - العدد الثالث - الاصدار السادس-شهر يونيو
P. 14
???? ???? ???????
????? ???????? ??????? ??????????
?????? ??? ?????
????? ??? ???? ?????? ?????? ?????????
??????? ?( BERT???????? ???? ???? ????? ??????? ???? ??????) ?? ? Google???? ? 2018???? ???? ???? ????? .???? ???? ???? ?? ?? ??????
?????? ?????????? (?????? ??????? ????????) ??????? ??????? ???????? .?????? ?????? ?????? ??????? — ????? ???????? ?????? ? ????? ?? ????? ?? ?? —?
????? ????? ?? ???? ????? ?????? ??????? ?????? ???????? ??????? ???????? ?? ? ???????.??
?? 4.2????? ???? ???? :?????? ?GPT??
??????? ????? ?( GPT????? ???? ???????? ????? ???? ???? ??) ?? ? OpenAI???? ???? ???? ???? ?? ??????? ?????? (?? ?????? ??? ??????)?.??
???? ???? ??????? ???? ?? ?? ?????? ?????? ?????? ?? ???????? .?????? ??? ??????? ????????? ????? ??????? .???? ????? ?175( GPT-3??
?????? ?????) ??????? ??? ????? ???? ?????? ??? ???? ???? ???? ????? ????? ?????????? ?????? ?????? ??????? ????? ???????? ?????? ???????.??
?? 4.3????? ???? ????? ???? ?? ??? T5 :???BART??
????? ?( T5??? ???? ????? ?? ??? ??) ?? ? Google??? BART??? ? Facebook Research???? ???????? ????? ????? .?????? ??? ????????
??? ???? ??????? ??????? — ??????? ???????? ?????? ???????? ???????? — ?????? ?? ???? ? NLP?????? ?? ?? ??? ???.??
?? 4.4??? ?????? ?????? (?)ViT??
??? ????? ?? ?????? ?????? ??? ? ?2020????????? ???? ???? ???? ?? ????? ??????? ?????? ?????? ??? ????? ??? ??? ????? ???????? ?? ??????
???? ??? ????????? ????? ????? ????? ?? ????? .???? ???? ? ViT????????? ????? ?????? ?? ????? ????? ???? ? ?????? ???????
??????? ??????? ?? ??????? ??????? ??????? ??????? ???????????.??
?? 4.5????? ?????? ??? ???????
????? ???? ????? ???????? ?????? ??????? ???????? ?? ??? ????????? ?? ??? ????? ????????? ????? ???? ??? ???? ?? ?????????? ????????
??? ??? .?????? ??????? ???? ???? ??? ?????? ??? Longformer??? BigBird??? Performer??? Flash Attention???? ??????? ?????? ??? ??????
?????? ???????? ?????? ????? ??? ??? ??? ???????.??
?? .5???????? ?????? ?????? ???????
??? ????? ??????? ???? ???? ?????? ???? ?? ??????? ???????/?????? ??????? .???? ???? ????? ???? ?????? ??????? ???? ?? ?? ????? ????? ???
????????? ??? ?????????? (?????? ??? ?????? ??? ?????) ???????? ?????? ????? ?????? .????? ??? ?????? ?? ????? ?? ?????? ?? ???? ?? ?????
??????? ?????????? ????????.??
????? ???? ???????? ????? ?????????? ????? ?????????? ????? ??????????
???.?? ???? ?????? ???.???.?? ?/?????? ???????? ??? /????? ????? ???/?????? ????????
???/????? ?????
?? ?/?????? ?????
??Address: New Beni-Suef City. Beni-Suef. 62111 Web Site: WWW.fci.bsu.edu.eg??
??Email: fci@fci.bsu.edu.eg?? ??Telephone/Fax: 082 2246796??
