ctx length 2048.
#1
by
Prakh24s
- opened
Hey, thank you for the amazing work.
I was trying to generate some svg, and I saw that the model copuld not complete the svg code geenration.
On a deeper look, I found that, as stated in your paper, the average tok length for GPT2XL model is 2297, which is larger than the ctx/length of this model (2048). Could you please let me know what I am doing wrong?