Quick look
In my day job, I’m a data engineer and writer passionate about building scale-able data systems. I grew up fascinated by how information flows and how we can use data to make better decisions.
My focus is on designing and implementing modernized data platforms, bridging the gap between engineering and analysis. I work with such open-source and cloud technologies to bring data pipelines, real-time streaming, and machine learning workflows into production.
Over the years, I have worked on various data engineering projects, helping organizations build automated ELT workflows, modern data warehouses, and data governance solutions. I also enjoy writing technical content, sharing my experiences, and contributing to the data community through open-source projects DataPods, blogs, and Data Engineering Handbook.
I’m actively exploring serverless data platforms and their impact on modern data engineering. My recent work includes StringX (failed), an open data learning toolkit, and DataPods, an initiative to enable the Modern Open-Source Data Stack.
I strongly believe in continuous learning and sharing knowledge. Through my Data Engineering Handbook and various online resources, I help engineers navigate the evolving data landscape. I also engage with the community through meet-up, technical talks, and coaching.
What I am doing now
Designing and building the thinking system for better learning, topics are related to Data Engineering, Platform Ops:
- Gen AI Product (Not public): Support Founder to launch company, building and consulting data engineering workflows, involving into development part.
- Practicing data projects (completed): Check out the 9 weeks of data with LongDDL’s Data Camping
- Development Open Data Solution: Building a data toolkit for data engineering
- StringX (failed) - enabling data with Modern Open-Source Data Stack
- DataPods (on-going) to simplify data workflows
- Technical Blogs: Sharing technical posts at Blogs and on YouTube
- Maintain Second Brain (life-long): Developing a digital knowledge management system to enhance productivity and creativity at Brain
- Maintain Data Engineering Books (1/2 going to be released): Helping others organize and search daily data engineering questions
- Data Engineering Handbook helps engineers navigate the evolving data landscape.
- Starting (mini) data platform Serverless: Serverless Data Platform (Starting Kit)
- Community Engagement (hosting): Hosting local meetups for the data engineering community, supporting engineers, and connecting via our Discord community
Long’s Universal
Know more about me via longdatadevlog DNS:
- Data Engineering Handbook
- Open Source Data Projects - DataPods
- Lead Data Engineer (Full-time)
- Community Committee
- Data Engineering Blog
- Used & Hobbies
- Podcasts & Talks
- Products & Lifestyle
If I am not here, you can use this page https://longddl-io.vercel.app for access everything.
Professional Services
Beside the free resources you can find here, I also offer professional services to help business, unlock the project struggles, and indivituals in data engineering. Check Out Services and the Principles of Me for details.
About the failed project
I don’t hesitate to talk about the failure in development…
- With StringX, I was trying to enable data expert to use the SaaS for easier doing data works such as data ingestion, data transformation, and data analytics. It was a great idea, but it was not a good idea to build a SaaS for data engineering. I failed because I lack of knowledge about the world that many principal engineering TEAM(s) are working on. It is the Market Research Failure ?, yes it could be, but I was exploring and making a lot of mistakes. But it’s fine, I learned a lot from it and worth it.
Distributed under tree-map