1. Open-Assistant
  2. model
  3. pretokenizer
Name
..
configs
README.md
create_hf_tokenizer_config.py
indexed_dataset.py
pretokenize.py
requirements.txt
tokenizer.py