JesseTong<p><span class="h-card" translate="no"><a href="https://social.linux.pizza/@PeterMotte" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@<span>PeterMotte</span></a></span> <span class="h-card" translate="no"><a href="https://mastodon.gamedev.place/@lianna" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@<span>lianna</span></a></span> I don't think a machine translation model would need to scrape all the web like with LLMs or image generation models, because they are usually encoder based rather than decoder based like <a href="https://mastodon.world/tags/LLMs" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>LLMs</span></a> (like <a href="https://mastodon.world/tags/ChatGPT" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>ChatGPT</span></a>), encoder models are smaller and often require less data. Also <a href="https://mastodon.world/tags/MachineTanslation" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>MachineTanslation</span></a> requires high quality translated sentences pairs by languages so scraping the web blindly will never work well for them.</p>