Spaces:

lucalp
/

blt-entropy-patcher

Running on Zero

App Files Files Community

blt-entropy-patcher / bytelatent /data /iterators

Ctrl+K

Ctrl+K

12 contributors

History: 16 commits

par-meta's picture

Update ppl evals to work with blt model, in addition to entropy model (#82)

083656c unverified 3 months ago

__init__.py

53 Bytes

Initial commit 6 months ago
abstract_iterator.py

994 Bytes

Update iterator inheritance, pass file format args, limit iterator (#63) 4 months ago
arrow_iterator.py
12.9 kB

Reduce per file resources arrow uses (#77) 4 months ago
dev_iterators.py

2.3 kB

Update iterator inheritance, pass file format args, limit iterator (#63) 4 months ago
limit_iterator.py

1.46 kB

Update iterator inheritance, pass file format args, limit iterator (#63) 4 months ago
looping_iterator.py

1.07 kB

Update iterator inheritance, pass file format args, limit iterator (#63) 4 months ago
multiprocess_iterator.py

16 kB

Let process start before yielding preloaded prefetch buffer, avoid needlessly losing buffer in edge cases (#75) 4 months ago
packing_iterator.py

13.4 kB

Update ppl evals to work with blt model, in addition to entropy model (#82) 3 months ago
preprocess_iterator.py

4.27 kB

Update iterator inheritance, pass file format args, limit iterator (#63) 4 months ago
sampling_iterator.py

2.34 kB

Update iterator inheritance, pass file format args, limit iterator (#63) 4 months ago
sequence_iterator.py

6.91 kB

Reduce per file resources arrow uses (#77) 4 months ago
test_arrow_iterator.py

3.5 kB

Update iterator inheritance, pass file format args, limit iterator (#63) 4 months ago
test_iters.py

3.02 kB

Update iterator inheritance, pass file format args, limit iterator (#63) 4 months ago
test_limit_iterator.py

1.37 kB

Update iterator inheritance, pass file format args, limit iterator (#63) 4 months ago
test_packing_iterator.py

8.98 kB

Pass mask in packing_iterator, correctly handle last batch, fix masking (#65) 4 months ago