From Paper to Structured Data: Building a Lightweight OCR Profile Builder

Directly converting physical cards into database rows

Manual profile creation during onboarding frustrates users, but we can eliminate this friction entirely by chaining raw image extraction with LLMs to translate simple photos into clean, structured database rows. In my experience building the onboarding flow at Kareti, we wanted to bootstrap digital profiles by letting users snap a photo of physical business cards or flyers. We chained a Vision API with a structured LLM call to process raw imagery into an active database entity. This approach bypassed manual input, turning physical artifacts into interactive digital accounts. Users expected immediate setup, and raw visual processing made that possible. This eliminated the need for manual copy-pasting, which historically caused high dropout rates during signup.

Capturing physical media means accepting unstructured noise. The pipeline extracts text, cleanses formatting, and populates schemas containing core details like phone numbers and addresses, as well as operating hours. This design bridges physical media and database tables. By converting raw visual data into structured representations, we saved our users from typing long strings on mobile keyboards. This automation turns a paper card into a populated schema in seconds, transforming raw pixels into structured assets. It converts chaotic real-world inputs into highly organized data structures.

Why traditional OCR is not enough for messy layouts

Standard OCR solutions rely on a step-by-step workflow of image acquisition, preprocessing, segmentation, and subsequent classification with post-processing. While this works for clean linear text, it does not understand context. It translates pixels into raw characters without understanding what those characters mean. A phone number looks the same as a serial number to a basic classifier. The technology lacks any inherent semantic awareness of how physical layouts translate to business concepts.

Traditional OCR systems provide raw text, but they force developers to write custom regular expressions or manual parsers to rebuild tables and structural hierarchies. If a business card places the phone number in an unusual corner or uses vertical text, standard regex parsers fail. These rigid parsers require constant updates when layouts change. When a flyer has multi-column layouts, traditional extraction outputs a chaotic stream of jumbled sentences. This messy output fails to represent the original design. The developer must write complex positional logic just to determine which phone number belongs to which contact name. This brittle code fragilely breaks whenever a user uploads an image with slightly shifted margins.

The pipeline architecture: OCR first, then parse

In a multi-page document pipeline, OCR typically dominates processing time, scaling linearly with the number of pages. For example, processing a document can take several seconds per page, while the parsing step runs once over stitched text. Running OCR first ensures we only send text to the language model, which keeps token usage manageable. This separation of concerns speeds up processing significantly. The initial optical processing scales with physical document size, while semantic parsing stays efficient because it only reads the extracted characters. This architecture avoids overloading the language model with raw image processing tasks.

We must first perform image acquisition, run per-page OCR, and then stitch the text in page order. This stitched output transforms visual segments into a unified, lightweight text document. The structured parser then processes this single text payload in a single, sublinear pass, reducing API costs. This architecture isolates the heavy computer vision work from the semantic reasoning work, ensuring stable performance. By avoiding multiple redundant LLM calls across every individual page, we save both time and money. It also provides a clear debugging point where you can inspect raw text before parsing begins.

Designing the LLM schema for profile extraction

At Kareti, we defined explicit JSON schemas for our business card extractor. The schema required keys like business name, phone number, and address. We used clear JSON property descriptions to guide the LLM's parsing decisions. In my work on this feature, we found that letting the LLM output freeform text always resulted in extraction errors. The system needs a strict contract to communicate with downstream database models. By defining precise data structures, we prevent formatting variations from breaking database writes.

Defining descriptions within schemas is the fastest way to achieve consistent output. Instead of writing long prompt instructions, we specify validation boundaries. This minimizes omission errors and guarantees standard database types. If a field is missing from the physical card, the schema should allow a null value rather than letting the LLM invent data. This control prevents hallucinated contact details from polluting your databases. It forces the language model to remain honest when processing partial or damaged documents.

{
  "type": "object",
  "properties": {
    "business_name": {"type": "string"},
    "phone": {"type": "string"},
    "address": {"type": "string"},
    "hours": {
      "type": "object",
      "properties": {
        "open": {"type": "string"},
        "close": {"type": "string"},
        "days": {"type": "array", "items": {"type": "string"}}
      }
    }
  },
  "required": ["business_name", "phone", "address"]
}

Dealing with skewed angles, bad lighting, and foreign abbreviations

In our Kareti production setup, we found that users rarely upload studio-quality images. Most pictures are skewed, dimly lit, and shot at off-center angles. Our LLM pipeline had to handle regional dialects and localized abbreviations. Real-world users take quick snapshots on the street or in busy offices, resulting in shadows that obscure text characters. The extraction engine must be resilient enough to infer characters under these conditions. If the physical text is partially unreadable, the system must degrade gracefully. This resilience determines whether your onboarding flow succeeds in the wild or fails under normal lighting conditions.

We had to translate French and Arabic business abbreviations, such as 'TND' for currency or 'Toute la semaine' for operating hours. We resolved this by including precise mapping definitions inside the LLM prompt. The system normalizes these terms into ISO currencies and standardized schedules. For instance, 'Toute la semaine' becomes a structured array of seven active days. This mapping ensures our database receives clean, queryable values instead of localized jargon. It handles language blending without requiring manual translation steps. This step is essential when operating in multilingual regions where users mix business terms.

Choosing between hosted Vision APIs and lightweight local models

For teams wanting to avoid expensive SaaS dependencies, lightweight local multimodal models offer an alternative. Models like GLM-OCR run on consumer hardware at just 0.9 billion parameters, producing direct semantic structures. Running these models locally keeps sensitive user data within your private infrastructure. It also eliminates network latency to third-party endpoints. However, maintaining this infrastructure demands continuous monitoring and GPU provisioning. If you lack in-house machine learning engineering talent, hosting these models can become a major operational burden. You must balance the physical cost of hosting with the simplicity of direct api consumption.

Conversely, hosted document processing APIs like Extend provide pre-trained agentic optimization, schema versioning, and built-in evaluation frameworks. These APIs reduce deployment times from weeks of manual tuning down to days. They handle various layouts, poor image quality, or messy handwriting without requiring custom model pipelines. For startups looking to launch features quickly, managed APIs remove the burden of scale. The developer simply pays for what they use, bypassing the complexity of managing hardware. This choice allows smaller product teams to ship document understanding capabilities in a fraction of the time.

Using structured output to control LLM costs

Structured outputs stream processing pipelines by enforcing strict limits on output format. This approach minimizes the total tokens generated by eliminating conversational preambles. When an LLM does not need to write pleasantries like 'Here is your parsed data', it spends fewer tokens. This directly translates to lower bills when processing thousands of documents per month. In my experience, restricting the output to a strict JSON structure cut down redundant tokens dramatically, making the entire pipeline more economically viable. Every token saved on whitespace or conversational text reduces downstream processing latency.

Optional compaction techniques remove whitespace and unnecessary characters from the JSON payload. In high-volume onboarding environments, removing this bloat regulates inference costs and maintains low latency. By minimizing whitespace, you optimize the network payload and ensure faster processing. This optimization helps keep your API costs predictable and your application responsive. When every millisecond counts during user sign-up, saving these tokens makes a visible difference in client-side loading states. It keeps your system fast even on slower mobile networks.

My checklist for building your own document ingestion pipeline

To build your own ingestion pipeline, start by establishing clear schema definitions. You should never pass raw image data directly to an LLM without an intermediate text extraction or formatting layer if you want to optimize costs. In my development of the onboarding flow at Kareti, we learned that a multi-stage approach yields the most reliable results. This process ensures that errors are caught early before they hit the database. It allows developers to test character extraction separate from parsing logic.

Ensure your pipeline handles layout skewness and translates localized terminology. Finally, implement an evaluation framework to track parsing accuracy over time. This checklist provides a practical guide to launch your system successfully:

First, separate image text extraction from semantic schema parsing. Second, implement strict JSON schema validation to prevent LLM hallucinations. Third, add local normalization dictionaries for regional abbreviations. Fourth, set up automated image rotation to correct skewed angles. Fifth, monitor token usage to keep inference costs low. Sixth, design your system to handle missing fields gracefully by utilizing nullable JSON properties.