Apache Spark utilizes different amount of RAM in different RAM configurations, is this expected?

[KhanjanD]

Hello @maziyarpanahi

Actually, I am trying to load the model t5_base in Java in 2 machine configurations:

1. With 8 GB RAM (windows 7)
2. With 16 GB RAM (windows 8 )

The model is of size ~950 MB.

Problem

I have kept all configurations the same in both instances (8GB and 16GB), all XMX and XMS are the same in both.
But the thing is, in 8GB instance it gets loaded under 7.5 GB RAM while in 16GB instance the same model with the same code it takes around 13GB of RAM.

Now, because of this I am not able to make sure if this will work on the client machine or will crash.
Is there any special configuration that I will need to set to limit its RAM usage?

Does it use RAM respective to the availability?

Spark Version: 3.4.1
Spark NLP Version: 4.3.2

modelPath = "t5base model dir path"
model = (T5Transformer) T5Transformer.load(modelPath);

Please help, thanks

[maziyarpanahi]

Hi,

XMX and XMS don't really mean anything to Apache Spark session/application. Spark has its own memory configs which in a standalone setup is either ours which is by default 16G or whatever you set for spark.driver.memory.

The second thing to keep an eye on is spark.memory.fraction config. So it uses a fraction of given available memory https://spark.apache.org/docs/latest/tuning.html#memory-management-overview (it is true that it will use GC to clean it up if not required, but you can control how much to be used exactly if you want them to be exact)

If you provide how you launch/start your SparkSession I can make sure whether it's default Apache Spark configs like spark.memory.fraction and others or there is something you are passing along as well

[KhanjanD]

Thank you for your response. I have pasted my SparkSession Configs below:

SparkConf sparkConfig = new SparkConf();
sparkConfig.set("spark.memory.fraction", "0.6");
SparkSession spark = SparkSession.builder().appName("SparkNLPTextToSQLConverter").config(sparkConfig)
.master("local[*]").getOrCreate();

The problem occurred with the default configuration, also set the memory config parameter but we are still facing the same issues with consuming all memory (RAM).

[KhanjanD]

Hello @maziyarpanahi waiting for your suggestion for the above configuration.

[maziyarpanahi]

Spark NLP does not control nor has a way to even access available memory.

If you want your Spark application to consume up to a certain memory, you must mention it explicitly:

spark = SparkSession.builder
    .appName("Spark NLP")
    .master("local[*]")
    .config("spark.driver.memory", "16G") <----- HERE, Spark/JVM will manage the available memory as they see it fit
    .config("spark.driver.maxResultSize", "0")
    .config("spark.kryoserializer.buffer.max", "2000M")
    .config("spark.jars.packages", "com.johnsnowlabs.nlp:spark-nlp_2.12:5.1.1")
    .getOrCreate()

Apart from this, the rest is between JVM and Apache Spark which I believe you can have better answers looking into the Spark configs and ask in their Dev/User mailing lists.

[KhanjanD]

Hello, thanks for the response.
I have a quick question regarding the memory management in JSL.

Actually, I have run the t5_base which is around 850MB in python using the transformers library which takes around 1.2GB RAM to fit and inference through -- also t5_small which is around 250MB which takes around 500MB of RAM.

Now, running the same models of the similar sizes in JSL (available through its model collection) takes 4-5GB with small and around 8GB for the t5-base.

Why such a huge RAM consumption? I am only using T5transformer.pretrained() or T5transformer.load() to load the model, am I doing it wrong?

Thanks