Hello everyone, I'm learning how to built a bot that uses rag for data retrieval and and there is also some pdf formatted data that I'm going to use in this. But the problem is rag models can't read pdf data so I have to structure the data. Converting the pdf into CSV manually is time consuming. Can someone please tell me how to do it in fast and efficiently way
if the pdf file contains data in tables you can try this package
you can read your pdf file and convert them to csv and other formats as well
https://pypi.org/project/tabula-py/
- you need to have java 8+ installed fot it to work
does the model accept images? are those diagrams in image format?
anyways you can read the docs of it maybe it had some methods or functions to extract images as well
if it didn't you can try some other packages like pypdf and for the diagrams I don't really know, it depends on what format they are attached in to the pdf.
No the bot doesn't accept images, but to clearify some concepts of physics it must provide some image of a diagram. For data extraction from pdf I tried pdfplumber for text and tables, and pymupdf for images and link. When I opened those image they are tiny like square dot shaped images. I have no idea is this how they are supposed to look
hi
class Questions:
def __init__(self, question: str, correct_answer: str, answers: list[str]):
question.self = question
correct_answer.self = correct_answer
answers.self = answers
def __dict__(self):
<seems like I should put sth here>
def indexing(self):
pass
Questions("where?", "1", ["1. up", "2. down", "3. left", "4. right"])
I have no idea why cant I put a list as an attribute it gives me this error:
question.self = question
^^^^^^^^^^^^^
AttributeError: 'str' object has no attribute 'self' and no __dict__ for setting new attributes
I have no idea what that dict is, asked some language models, some said I should add it, some said I shouldnt, all I need to do is to put a list as the third attribute so I can start working on other parts of the program
self refers to the instance of class Question. It is not an attribute of strings. You shouldn't do anything with __dict__. Just fix __init__ and assign values to self.whatever.