---
esco_uri: http://data.europa.eu/esco/skill/f9670490-8aa4-4540-b121-d440a8294aab
esco_label: "develop data processing applications"
relation: essential
onet_soc: 15-2051.00
source: model-knowledge
confidence: high
qa_count: 0
generator: gemma3:27b (prompt-designed and spot-checked by Claude)
generated: 2026-07-10
---

# develop data processing applications — Data Scientist

As a Data Scientist, 'develop data processing applications' isn’t about *building* full-blown software products; it's about rapidly prototyping and deploying code to transform raw data into analysis-ready formats. Daily tasks involve writing scripts (often in Python or R) to clean messy datasets – handling missing values, outliers, inconsistent formatting – then performing feature engineering: creating new variables from existing ones that improve model performance. Think automating ETL pipelines (Extract, Transform, Load) for recurring data sources, building custom functions to handle specific data quirks, and sometimes wrapping these into reusable modules or microservices.

Typical tools go beyond basic scripting. You'll frequently use libraries like Pandas, NumPy, and scikit-learn in Python; dplyr and tidyr in R.  For larger datasets, you’ll leverage distributed computing frameworks like Spark (often with PySpark) or Dask to parallelize processing. Cloud platforms (AWS, Azure, GCP) are common for scaling these applications. Version control (Git) is *essential* – treating your data transformation code as production-level software even if it's 'throwaway'.

A key pitfall is over-engineering. Data Scientists need to deliver insights quickly; spending months building a perfectly optimized, fully generalized application isn’t usually the priority.  Another is neglecting data quality checks *within* your processing pipeline – assuming input data will always be clean leads to unreliable results. Poorly documented code and lack of testing are also common issues that hinder reproducibility and collaboration.

Good looks like: well-commented, modular code; automated tests verifying data transformations; clear documentation outlining the purpose and assumptions of each script/module; efficient handling of large datasets; and a focus on delivering clean, reliable data to downstream analysis or modeling tasks. It's about being a pragmatic programmer focused on *data* problems, not just software engineering best practices (though those are important too!).

## Weitere Anreicherung

Stage-2 source for future practitioner grounding: arXiv cs.LG/stat.ML (ML preprints).
