Ingestion Examples

Code example

The following example takes an HTML file, applies an HTML specific transformer, to finally generate a Text document:

from ingestum import sources
from ingestum import transformers

source = sources.Text(path="tests/data/test.txt")
document = transformers.TextSourceCreateDocument().transform(source)
document = transformers.TextSplitIntoCollectionDocument(separator="\n\n").transform(document)

print(document)

More examples

You can find annotated ingestion examples here. If you’re new, we’d recommend starting with Example: Text Files to get a taste of how ingestion works at Sorcero. Note that the tutorials are designed with the assumption that you have a basic understanding of the core concepts of ingestion (sources, documents, transformers, conditionals, pipelines and manifests). If you need a refresher, check out Ingestion Basics.

• Example: Text Files
• Pipeline Example: Text Documents
• Example: Spreadsheets
• Pipeline Example: Spreadsheets
• Example: Webpages
• Pipeline Example: Webpages
• Example: PDF Documents
• Pipeline Example: PDF Documents
• Example: XML Files
• Pipeline Example: XML Files