Redefining Data Engineering with AI (Agentic Engineering Your Way to Data Democracy)

While much of data engineering still means writing repetitive code, stitching together orchestration tools, and manually validating systems, the emergence of AI-assisted development is fast transforming this work. Redefining Data Engineering with AI introduces the concept of agentic engineering, a structured approach in which engineers express intent in natural language, and AI generates pipelines, documentation, test suites, and monitoring. Engineers remain in control, focusing on big-picture design, governance, and quality while delegating routine implementation to AI.

This go-to guide uses realistic case studies to show how large language models can support each phase of a data project. Through a guided build of a patient search platform, you'll learn how to design systems that integrate AI responsibly and effectively. Each chapter leads you through a practical step in the lifecycle, illustrating exactly where AI adds value and where human input is most crucial.

• Describe business goals and translate them into pipeline-ready assets
• Generate code, test cases, and documentation using structured prompts
• Integrate LLMs and governance into modern data workflows
• Automate observability and reduce operational drift with AI-driven monitoring
• Create secure data products while preserving compliance and quality
• Adopt a repeatable, intent-driven framework for scalable data practices