Page 13 - العدد الثالث - الاصدار السادس-شهر يونيو
P. 13
???? ???? ???????
????? ???????? ??????? ??????????
?????? ??? ?????
????? ??? ???? ?????? ?????? ?????????
???? ??? ???? ???????? ??? ??? ?? ????? ???? ???????? ????????? ???????? ???? ??? ???? ??????? ?????? ???????? ? ???? ???????? ?????????
?? softmax??????? ?? ????????? ???? ????? ?? ?????? ??????? .???? ??? ??? ??????? ? ??????? ?????????? ??????? ???? ????? ???????
?????? ????? ?????? ??? ???? ???? ???????.??
?? 3.3????????? ????? ???????
???? ?? ?? ????? ????? ?????? ?????? ???????? ???? ?????? ???????? ????? ??????? .??????? ???????? ?? ?????? ????????-???????-??????
??????? (? 8???? ? 16???? ??)? ??????? «??????»? .????? ??? ???? ??? ???? ??????? ?????? ?????? ?? ?????? — ??? ???? ???? ???
????????? ??????? ????? ???? ??? ?? ??????? ???????? .?????? ?????? ???? ?????? ??????? ??? ?? ??? ?? ???? ? ??????.??
?? 3.4?????? ??????? ? ????????
???? ?????? ??????? ????????? ?????? ?? ???? ?? ????? ???? ???? ???????? ??? ???? ????? ??????? ??????? .??????? ??? ?????? ???
??????? ??? ?? ??? ??? ?? ???? ????? ??? ???? (? ReLU???? ? GELU???? ??) ??????? .??? ???? ??? ?????? ????? ??? ???? ?? ?? ???? ???????????
?????? ????? ???? ??????? ????? ?????? ?? ????? ??????? ???????.??
?? 3.5?????? ?????? ???????? ?????????
???????? ?? ???? ????? ?? ???? ???? ????? ??????? ???????? ????? ????? .????? ??????? ???????? ???????? ??? ????? ??????? ??? ??????????
???? ????? ????? ???????? ?????????? .????? ????? ?????? ? ??????? ??????? ?????? ????????? ??? ???? ???????? .????? ??? ???????? ?? ???
?????? ???? ?????? ???? ?? ?? ?????? ???????.??
?? 3.6????? ???? ?????-????? ?????
?????? ???? ???? ????????? ???? ???? ????? ?????? ?? ????????? ????????? .???? ???? ???????? ????????? ???? ???? ????? ?????? ?????? ?? ??????? ???????
????? ????? ???? ?? (??? ???????? ?????? ?????????) ???????? ???? ???? (??? ???????? ????????)? .???? ??????? ????? ????? ???????? ???????? ?????????
??? ? ???????.??
?? .4?????????? ???????? ???? ?????
???? ?????? ? ??????? ??? ???? ???????? ??????? ????? ???? ???? ??? ????? ????? ????? ???????:??
?? 4.1????????? ????BERT :??
????? ???? ???????? ????? ?????????? ????? ?????????? ????? ??????????
???.?? ???? ?????? ???.???.?? ?/?????? ???????? ??? /????? ????? ???/?????? ????????
???/????? ?????
?? ?/?????? ?????
??Address: New Beni-Suef City. Beni-Suef. 62111 Web Site: WWW.fci.bsu.edu.eg??
??Email: fci@fci.bsu.edu.eg?? ??Telephone/Fax: 082 2246796??
