Data extraction from a research article (an electronic pdf with highly unstructured data)

My research area is applied machine learning in material science. I am struggling for an algorithm which could retrieve the desired data values (categorical and numerical both) from the research article (an electronic pdf document which is highly unstructured). There are thousands of such documents (pdfs) from which the data needs to be extracted which is a time intensive task. Some pdf may carry data in graphs while others in table or text. Kindly guide me through the process using which I can efficiently extract the data.

[dpb]

Essentially an impossible task -- the data/figures in a pdf file are not stored in a retrievable format other than by interpreting/rendering the pdf document itself.

Answer 1

[Image Analyst]

Try this:

https://www.mathworks.com/matlabcentral/fileexchange/?term=tag%3A%22digitize%22

[Aditi Mahajan]

I have attached a research article from which I need to extract the materials studied, processes deployed, material characterization type (all these given in text); sequence type (given in table); extracting the mechanical characterization (given in figures).

Is this possible in one go? Or do I need to do it in fragments using codes? Or some collection needs to be done manually?

[dpb]

@Image Analyst is pointing out that the "converters" actually use OCR to recognize and convert pdf content to text. Then, you'll have to have code to find the particular code words of ineterst.

That's only the text portion; tables and images in my experience weren't converted to raw data but simply embedded into the document as objects. That might get you at least part of the way, but it's not going to be anything simple to do for a generic collection of papers.

See <acrobat/online/convert-pdf.html> and Google is your friend to find alternates outside Adobe...altho your uni probably has a site license.

This really isn't much a MATLAB Q?

Answer 2

[Image Analyst]

I doubt thousands of articles would all be in this format/style. You might be able to get the text and numbers out but it could be tough to automatically figure out which numbers are in a table and what they mean. For the plots, you might just need to convert them to images and then use one of the File Exchange submissions. But even then I imagine it's going to require a lot of manual processing.

Maybe you could just use Amazon Mechanical Turk to hire a bunch of cheap global workers to do it for you.

https://www.mturk.com/