---
esco_uri: http://data.europa.eu/esco/skill/c5e8abde-d2ba-4e8e-a65e-720b71180666
esco_label: "unstructured data"
relation: optional
onet_soc: 15-2051.00
source: model-knowledge
confidence: high
qa_count: 0
generator: gemma3:27b (prompt-designed and spot-checked by Claude)
generated: 2026-07-10
---

# unstructured data — Data Scientist

As a Data Scientist, 'unstructured data' isn’t an abstract concept – it's *most* of what you'll spend your time wrangling. Think text from customer reviews, social media posts, emails, legal documents, audio recordings of call centers, or images/video feeds. The ESCO definition is spot on: this data lacks neat rows and columns. Daily tasks involve extracting meaningful signals – sentiment analysis from text, identifying objects in images, transcribing & analyzing speech, or finding key themes within large document sets. It's rarely about 'clean' datasets; it’s about building pipelines to *make* them clean(er) enough for modeling.

Tools are diverse: Python (with libraries like NLTK, spaCy, OpenCV, transformers), cloud-based NLP/vision APIs (Google Cloud Natural Language, AWS Rekognition, Azure Cognitive Services), and increasingly, specialized LLMs.  You'll use techniques like regular expressions, topic modeling (LDA, NMF), word embeddings (Word2Vec, GloVe, BERT), computer vision algorithms, and potentially even manual annotation/labeling for training supervised models. Expect to spend significant time on data cleaning, pre-processing, and feature engineering – often 70-80% of the project.

Pitfalls are numerous: bias in text data (reflecting societal biases), noisy or incomplete transcriptions, difficulty scaling manual annotation, and the 'curse of dimensionality' with high-dimensional image/video features.  Overfitting to specific unstructured formats is common – a model trained on one type of customer review might fail spectacularly on another.  Also, be wary of relying *solely* on automated tools; human-in-the-loop validation is often crucial.

Good looks like: being able to rapidly prototype data extraction pipelines, choosing the right tool for the job (sometimes a simple regex is enough!), understanding the limitations of your models, and clearly communicating uncertainty. It's not about achieving perfect accuracy on unstructured data – it’s about delivering *actionable* insights despite its inherent messiness.  A strong Data Scientist can articulate what information *can* be reliably extracted and how that impacts business decisions.

## Weitere Anreicherung

Stage-2 source for future practitioner grounding: arXiv cs.LG/stat.ML (ML preprints).
