How to detect multiple tables in document image and convert into data frames

[PoornaSaiNagendra]

I want to store the text inside a table in a data frame but the output from doctr has no information about the position of the table starts and ends. In order to create a data frame, I require the number of tables present in a document image, number of rows and columns(structure of table) present. Also, there are a few other complexities like column head contains more than one sub-columns, borderless table, semi border table. Could you help me resolve the table structure? Thanks for any help you can provide.

[charlesmindee]

Hi @PoornaSaiNagendra,

Thanks for your interest in doctr!
For the moment we do not provide table detection/information extraction in doctr, we are working on it.
In the meantime, you can try to do it by yourself using the box coordinates: you can search for vertically align boxes and horizontally align boxes to reconstruct lines and columns for instance.

I hope this answer your question!

[felixdittrich92]

@PoornaSaiNagendra
I have currently a model to segment the table rows and columns in a document and to extract the information as dataframe i can provide it if you want to test it !? :) But note this is currently not optimal it takes ~4sec on CPU without OCR (In a later point i will investigate again and than i think we will also find a way into doctr :) )

[PoornaSaiNagendra]

Thank you for your great help. It could be beneficial for us if you could share it.

[PoornaSaiNagendra]

@felixdittrich92
I have tried the TableNet project that you suggested in #524 on my data.

Here I am providing the image for your reference.

Image source: Google Images

And I am getting this as output:

<class 'pandas.core.frame.DataFrame'>
0
0

Even I tried on bordered table images as well but I got similar kind of results.
.

[felixdittrich92]

@PoornaSaiNagendra
Oh yes i see this will not work the model was trained on marmot dataset (which contains only perfect scanned papers) ..
you can take a look at this but i think to detect borderless tables in mobilephone images will be also for this tricky 😅

[PoornaSaiNagendra]

@felixdittrich92
Yeah I even tried this as well. It is doing good on bordered tables but while changing it to cpu it is throwing error as it is using mmdectection as backbone which using uses cuda😅.