Understanding Robyn through simulated data

[yvonnewong-bellroy]

Hi everyone!
I’m interested in implementing Robyn MMM, but would like to assess the accuracy with some dummy data. I set up a simulation with random normal spending (6 channels) - and make some transformations (geometric decay and hill transformation) for some set level of theta, alpha and gamma (all the channels have different alpha, gamma but same theta). The transformed spend is then taken as caused revenue by each of the channels and summed together to be total revenue in my dataset. There is no background intercept/noise. I also added some random periods of increases/decreases in spending so that the spend/revenue ratio of each channel will differ enough for every pair of channels.

However, given this, the model is unable to tease out the right hyperparameters.

• When allowing the hyperparameters to vary within +-0.2 of the true value I specified, the model was able to converge on NRMSE and RSSD.DECOMP and performed well but the hyperparameters estimated were wildly incorrect (at least for theta based on the model onepager)
• When restricting the bounds of hyperparemeters to +-0.01 of the true value I specified in the dummy set, the model did not perform well at all, no convergence on NRMSE, and RSSD.DECOMP.

Additionally, given that I did not add any background revenue not caused by the channels, I would expect the intercept to be near 0, but it has been reflected as the largest driver of revenue in my dataset. I’m wondering if I’m interpreting the intercept in the waterfall decomposition plot wrongly?

How does the model teases out the hyperparameters and make the transformation and why it is not able to pick out the right values/with the right values not able to fit the data nicely?

[gufengzhou]

Good exercise. I can't say much without seeing your implementation. The way of transformation need to be identical.

[AdimDrewnik]

Did you make any progress with your simulations?