iiinova — where your digital self keeps evolving

OSCAR Project

@oscarproject.bsky.social

22 followers 7 following 6 posts

The Open Super-large Crawled Aggregated coRpus

Posts Replies Media

OSCAR Project @oscarproject.bsky.social · 10/08/2023

👀 We're working on many new features for you, currently we're focusing on improving language identification, so if you want to help or contribute, please join our community 💬 on Discord: t.co/toLKAPje4E

t.co

Join the OSCAR Project Discord Server!

Check out the OSCAR Project community on Discord - hang out with 365 other members and enjoy free voice and text chat.

000

OSCAR Project @oscarproject.bsky.social · 10/08/2023

✨ Colossal OSCAR 1.0 has also been made possible thanks to the continuous support of Inria, the ALMAnaCH and CommonCrawl. Specially thanks to the contributions of @ujj.bsky.social, Rua Ismail, @sobamchan.bsky.social, Sebastian Nagel and Benoît Sagot.

100

OSCAR Project @oscarproject.bsky.social · 10/08/2023

As Colossal OSCAR 1.0 is based on Common Crawl, our annotations are distributed under CC0 (Creative Commons Zero) license, however for the textual comments users agree to the Common Crawl Terms of use 📄 👉 commoncrawl.org/terms-of-use

commoncrawl.org

100

OSCAR Project @oscarproject.bsky.social · 10/08/2023

Colossal OSCAR 1.0 is just a partial annotation of the WET files of 10 Common Crawl snapshots, the original data is included only for convenience, and specially for researchers looking for data in lower resource languages. 🗣️

100

OSCAR Project @oscarproject.bsky.social · 10/08/2023

Colossal OSCAR 1.0 is by far our largest release so far, being almost 10 times as big as previous releases. We're still working on statistics and documentation so please bear with us while we finish these for you in the coming days and weeks. 🤓🧑‍🔬📊

100

OSCAR Project @oscarproject.bsky.social · 10/08/2023

📣 The OSCAR Project and DFKI are happy to announce the release of Colossal OSCAR 1.0 📚, which is now available on the Hugging Face Hub 🤗 at huggingface.co/datasets/oscar-corpu… Colossal OSCAR 1.0 was put together by @pjox.bsky.social as part of the OpenGPT-X collaboration.

151