Contact us

Archive: 2022 February

News
[Long Review] 'GShard': Scaling Giant Models with Conditional Computation and Automatic Sharding
Tutorials

[๊ธด ๋ฆฌ๋ทฐ] 'GShard': ์กฐ๊ฑด๋ถ€ ์—ฐ์‚ฐ ๋ฐ ์ž๋™ ์ƒค๋”ฉ์œผ๋กœ ๊ฑฐ๋Œ€ ๋ชจ๋ธ ํ™•์žฅ

Transformer๋ฅผ 100์–ต ๊ฐœ ์ด์ƒ์˜ ๋งค๊ฐœ๋ณ€์ˆ˜๋กœ ํ™•์žฅํ•˜๋Š” ๊ฒƒ์€ ๋น ๋ฅด๊ฒŒ ์ฒ ํ•™์ ์ด ๋ฉ๋‹ˆ๋‹ค: ๋ชจ๋“  ํ† ํฐ์— ๋Œ€ํ•ด ์ „์ฒด ๋„คํŠธ์›Œํฌ๋ฅผ ํ™œ์„ฑํ™”ํ•˜๊ฑฐ๋‚˜ ๊ฐ ํ† ํฐ์„ ์‹ค์ œ๋กœ ๋ด์•ผ ํ•˜๋Š” ์ „๋ฌธ๊ฐ€์—๊ฒŒ ๋ผ์šฐํŒ…ํ•ฉ๋‹ˆ๊นŒ?

2์›” 26, 2022ยท 1 min read
Triplets-like Russian Figure Skaters: Can Kullback-Leibler Divergence be Used Tell Their Difference?
Tutorials

์„ธ์Œ๋‘ฅ์ด์ฒ˜๋Ÿผ ๋‹ฎ์€ ๋Ÿฌ์‹œ์•„ ํ”ผ๊ฒจ ์Šค์ผ€์ดํ„ฐ๋“ค: Kullback-Leibler ๋ฐœ์‚ฐ์ด ๊ทธ๋“ค์˜ ์ฐจ์ด๋ฅผ ๊ตฌ๋ณ„ํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๊นŒ?

์Šคํ”ผ์ปค ์ ์‘์€ ์‹œํ€€์Šค-ํˆฌ-์‹œํ€€์Šค ๋ชจ๋ธ์„ ์ธ์ฝ”๋”์˜ ์–ด๋ ต๊ฒŒ ํš๋“ํ•œ ์Œํ–ฅ ํ‘œํ˜„์„ ์†์ƒ์‹œํ‚ค์ง€ ์•Š๊ณ  ์ ์‘์‹œํ‚ค๋ ค๊ณ  ์‹ค์ œ๋กœ ์‹œ๋„ํ•  ๋•Œ๊นŒ์ง€ ํ•ด๊ฒฐ๋œ ๊ฒƒ์ฒ˜๋Ÿผ ๋“ค๋ฆฌ๋Š” ASR ๋ฌธ์ œ ์ค‘ ํ•˜๋‚˜์ž…๋‹ˆ๋‹ค.

2์›” 19, 2022ยท 1 min read
Is Nathan Chen's 4 Flip scored by Mixture-of-Experts? Part 1: Switch Transformers: sparse MoE models
Tutorials

Nathan Chen์˜ 4ํšŒ์ „ ํ”Œ๋ฆฝ์ด Mixture-of-Experts๋กœ ์ฑ„์ ๋ฉ๋‹ˆ๊นŒ? Part 1: Switch Transformers: ํฌ์†Œ MoE ๋ชจ๋ธ

Nathan Chen์˜ 4ํšŒ์ „ ์ ์ˆ˜์— ๋Œ€ํ•œ ์šฐ์Šค๊ฐฏ์†Œ๋ฆฌ๋กœ ํ‘œํ˜„๋˜๋Š” ์ด ํ† ๋ก ์€ Switch Transformer๋ฅผ ์‚ดํŽด๋ด…๋‹ˆ๋‹ค. Google์˜ Mixture-of-Experts ํ™•์žฅ์— ๋Œ€ํ•œ ๊น”๋”ํ•˜๊ณ  ๊ณผ๊ฐํ•˜๊ฒŒ ๋‹จ์ˆœํ™”๋œ ํ•ด์„์ž…๋‹ˆ๋‹ค.

2์›” 12, 2022ยท 1 min read
Is Nathan Chen's 4 Flip scored by Mixture-of-Experts? Part 2: GLaM:Efficient Scaling of LMs with MoE
Tutorials

Nathan Chen์˜ 4 Flip์ด Mixture-of-Experts๋กœ ์ฑ„์ ๋˜๋‚˜์š”? Part 2: GLaM:Efficient Scaling of LMs with MoE

MoE ๋ฏธ๋‹ˆ ์‹œ๋ฆฌ์ฆˆ์˜ Part 2๋Š” GLaM์œผ๋กœ ์ดˆ์ ์„ ๋งž์ถฅ๋‹ˆ๋‹ค. Google์˜ 1.2์กฐ-ํŒŒ๋ผ๋ฏธํ„ฐ Mixture-of-Experts ์–ธ์–ด ๋ชจ๋ธ๋กœ, ์ œ๋กœ-์ƒท, ์›-์ƒท, ๊ทธ๋ฆฌ๊ณ  ํ“จ-์ƒท ๋ฒค์น˜๋งˆํฌ์—์„œ GPT-3๊ณผ ๋™๋“ฑํ•˜๊ฑฐ๋‚˜ ๋Šฅ๊ฐ€ํ•˜๋ฉฐ ํ† ํฐ๋‹น ์•ฝ 8%์˜ ํŒŒ๋ผ๋ฏธํ„ฐ๋งŒ ํ™œ์„ฑํ™”ํ•ฉ๋‹ˆ๋‹ค

2์›” 12, 2022ยท 1 min read
BERT Paper Reviewed from a Speech Perspective
Tutorials

์Œ์„ฑ ๊ด€์ ์—์„œ ๊ฒ€ํ† ๋œ BERT ๋…ผ๋ฌธ

BERT๋Š” NLP์—์„œ ๊ฑฐ์˜ ์†Œ๊ฐœ๊ฐ€ ํ•„์š” ์—†์ง€๋งŒ, ์Œ์„ฑ ์—”์ง€๋‹ˆ์–ด์˜ ๊ด€์ ์—์„œ ์ฝ์œผ๋ฉด ํ‘œ์ค€ "๋งˆ์Šคํฌ๋œ ์–ธ์–ด ๋ชจ๋ธ๋ง์ด ๋ชจ๋“  ๊ฒƒ์„ ๋ฐ”๊ฟจ๋‹ค"๋Š” ์š”์•ฝ๊ณผ๋Š” ๋‹ค๋ฅธ ๊ตํ›ˆ๋“ค์„ ๋ฐœ๊ฒฌํ•ฉ๋‹ˆ๋‹ค.

2์›” 6, 2022ยท 1 min read