I'm always getting n_tokens <= n_batch and then it fails running

[rossjackson]

Do you have any workarounds or suggestions on how to know if I am reaching max tokens per prompt?

Like a sample prompt is:

Can you summarize this for me:
    
Like the other planets, the Earth is a sphere. It rotates around its own axis slowly but continuously, and completes this rotation about once every 24 hours. As the Earth rotates, part of the Earth is facing the sun, and part of it faces away. This is why some parts of the world have day and night at different times.

There are a number of reasons why scientists know that the Earth is round, and not flat. It turns out that we can test many of these reasons out for ourselves, too. 

 If the Earth was flat, we would be able to look out and see very far, assuming that there were no buildings in the way. We could see another city that's hundreds of miles away. Or a ship sailing out to sea. The boat would stay in view no matter how far out to sea it went.
However, if we look out when standing on the ground, we can't see that far. We can't look out into the distance and see a town that's miles away. And a ship will slowly start to disappear from view as it gets farther away. This is because the Earth is round. Things in the distance are not visible because there is a curve to the landscape.

One way to test this is to watch the sunset. Find a spot where you can see the sunset, and also have a place close by where you can safely move higher, such as a tree or building. Or even start laying down and then stand up. On the ground, the sun will eventually disappear beyond the horizon. When this happens, quickly go to your higher spot. From there, you'll be able to see the sun again for a brief time. 

You have probably seen that when the sun is out, it causes a shadow to form if it hits something. It turns out that we can use shadows as another piece of evidence that the Earth is round.
Image of the Earth from the NASA DSCOVR satellite
The fact that the sun sets at different times in different locations lets us know that the Earth is a sphere. Image by NASA.

If the Earth were flat, then shadows would be the same length, regardless of location. This would mean that the sun is hitting all places on Earth at the same angle, resulting in the same shadow. However, it turns out that this is not the case. If we were to place two identical sticks in the ground in two different places on Earth, we'd find that the length of their shadows differs. This is because the Earth is round, and so the way the sun hits each location will be a bit different.

This also explains why the sun sets at different times in different locations. The sun is in a different position in relation to different locations. And so the timing of when it will lower in the sky and eventually disappear behind the horizon will change. This is also the reason that different parts of the world have different time zones. As one part of the world is facing the sun, another on the opposite side of the Earth is not. For example, the time in New York, United States is 12 hours different than the time in Beijing, China. For parts of the world that are closer together, the difference in time will be less.

During a lunar eclipse, the Earth passes between the moon and the sun. When that happens, the sun projects Earth's shadow onto the moon. When this happens, you can actually see the Earth's shadow if you look up at the moon. If you do this, you'll see that there is a round shadow, not a flat one.
A follow-up question might be whether the Earth could be flat and round, like a pancake. This would cause the shadow on the moon to be round, but the shape of the Earth to be flat. If the Earth were flat but rotating, its shape would be different for eclipses at different times of the day or night. Sometimes it would appear only as a line. The rest of the time it would appear like an oval or circle as it's spinning. However, we know that the shadow the Earth casts on the moon during an eclipse has always been circular, regardless of time of day. This supports the idea that the Earth is a sphere.

These are just a few testable ways we know that the Earth is round. Can you think of other ways that show the Earth is round and not flat?

[giladgd]

@rossjackson Try setting batchSize to the same value as the contextSize you have set on a LlamaContext.
The limit on a single prompt evaluation tokens count is the batchSize (including the tokens generated by the chat prompt wrapper)