bump up tokenizers version to 0.13.0 #248

lukas-blecher · Apr 13, 2023 · 3bd6f9a · 3bd6f9a
1 parent 9d99d1a
commit 3bd6f9a
Showing 1 changed file with 1 addition and 1 deletion.
diff --git a/setup.py b/setup.py
@@ -59,7 +59,7 @@
         'einops>=0.3.0',
         'x_transformers==0.15.0',
         'transformers>=4.18.0',
-        'tokenizers==0.12.1',
+        'tokenizers==0.13.0',
         'numpy>=1.19.5',
         'Pillow>=9.1.0',
         'PyYAML>=5.4.1',