Contact us

Archive: 2022 April

News
[Long Review] Fully Sharded Data Parallel: faster AI training with fewer GPUs
Tutorials

[๊ธด ๋ฆฌ๋ทฐ] Fully Sharded Data Parallel: ๋” ์ ์€ GPU๋กœ ๋” ๋น ๋ฅธ AI ํ•™์Šต

์ ๋‹นํ•œ GPU ์˜ˆ์‚ฐ์œผ๋กœ ์ˆ˜์‹ญ์–ต ๋งค๊ฐœ๋ณ€์ˆ˜ ์Œ์„ฑ ๋˜๋Š” ์–ธ์–ด ๋ชจ๋ธ์„ ํ•™์Šตํ•˜๋Š” ๊ฒƒ์€ ํŒŒ์ดํ”„๋ผ์ธ ๋ณ‘๋ ฌํ™”, ํ…์„œ ๋ณ‘๋ ฌํ™”, ๋˜๋Š” ZeRO ์Šคํƒ€์ผ์˜ ์˜ตํ‹ฐ๋งˆ์ด์ € ์ƒค๋”ฉ ์ค‘์—์„œ ์„ ํƒํ•ด์•ผ ํ•˜๋Š” ๊ฒƒ์„ ์˜๋ฏธํ–ˆ์Šต๋‹ˆ๋‹ค.

4์›” 23, 2022ยท 1 min read
[Short Review] Fully Sharded Data Parallel: faster AI training with fewer GPUs
Tutorials

[์งง์€ ๋ฆฌ๋ทฐ] Fully Sharded Data Parallel: ๋” ์ ์€ GPU๋กœ ๋” ๋น ๋ฅธ AI ํ•™์Šต

Fully Sharded Data Parallel์€ ๋ชจ๋“  ์Œ์„ฑ ๋ฐ ์–ธ์–ด ํŒ€์ด ๊ฒฐ๊ตญ ๋ฌป๋Š” ์งˆ๋ฌธ์— ๋Œ€ํ•œ Meta์˜ ๋‹ต๋ณ€์ž…๋‹ˆ๋‹ค: ๋” ํฐ ๊ทœ๋ชจ์˜ GPU๋ฅผ ํ”„๋กœ๋น„์ €๋‹ํ•˜์ง€ ์•Š๊ณ  ํ•œ ์ž๋ฆฟ์ˆ˜ ๋” ํฐ ๋ชจ๋ธ์„ ํ•™์Šตํ•˜๋Š” ๋ฐฉ๋ฒ•์€ ๋ฌด์—‡์ผ๊นŒ์š”?

4์›” 23, 2022ยท 1 min read
[Long Review] Deduplicating Training Data Makes Language Models Better
Tutorials

[Long Review] ํ›ˆ๋ จ ๋ฐ์ดํ„ฐ ์ค‘๋ณต ์ œ๊ฑฐ๊ฐ€ ์–ธ์–ด ๋ชจ๋ธ์„ ๋” ์ข‹๊ฒŒ ๋งŒ๋“ ๋‹ค

์ค‘๋ณต ์ œ๊ฑฐ๋Š” ์ง€๋ฃจํ•ด ๋ณด์ด์ง€๋งŒ, C4 ๋‚ด์— 61๊ฐœ ๋‹จ์–ด๋กœ ๋œ ๋ฌธ์žฅ์ด 60,000๋ฒˆ ์ด์ƒ ๋‚˜ํƒ€๋‚˜๊ณ , ์ด๋Ÿฐ ๋ง๋ญ‰์น˜๋กœ ํ›ˆ๋ จ๋œ ์–ธ์–ด ๋ชจ๋ธ์ด ํ›ˆ๋ จ ์„ธํŠธ ํ…์ŠคํŠธ๋ฅผ ๋ถˆ์•ˆํ•˜๊ฒŒ re

4์›” 16, 2022ยท 1 min read
[Long Review] CLAS: Deep context: end-to-end contextual speech recognition
Tutorials

[Long Review] CLAS: ๊นŠ์€ ๋ฌธ๋งฅ: ์ข…๋‹จ๊ฐ„ ๋ฌธ๋งฅ์  ์Œ์„ฑ ์ธ์‹

ASR ์ œํ’ˆ์„ ์ถœ์‹œํ•œ ์‚ฌ๋žŒ์ด๋ผ๋ฉด ๋ˆ„๊ตฌ๋‚˜ ๊ณ ์œ ๋ช…์‚ฌ์˜ ๊ณ ํ†ต์„ ์•Œ๊ณ  ์žˆ์Šต๋‹ˆ๋‹ค: ์—ฐ๋ฝ์ฒ˜ ์ด๋ฆ„, ๋…ธ๋ž˜ ์ œ๋ชฉ, ์•ฝ๋ฌผ ์ด๋ฆ„, ๋ชจํ˜ธํ•œ ์žฅ์†Œ ์ด๋ฆ„.

4์›” 9, 2022ยท 1 min read
[Long Review] Hurdles to Progress in Long Form Question Answering
Tutorials

[Long Review] ์žฅ๋ฌธ ์งˆ์˜์‘๋‹ต ์ง„์ „์˜ ์žฅ์• ๋ฌผ

์žฅ๋ฌธ ์งˆ์˜์‘๋‹ต์€ ๋ฆฌ๋”๋ณด๋“œ์—์„œ๋Š” ์ธ์ƒ์ ์œผ๋กœ ๋ณด์ด์ง€๋งŒ ์ž์„ธํžˆ ์‚ดํŽด๋ณด๋ฉด ๋ถˆ์•ˆํ•œ ๋ฒค์น˜๋งˆํฌ ์ค‘ ํ•˜๋‚˜์ž…๋‹ˆ๋‹ค.

4์›” 2, 2022ยท 1 min read