Job Title: Junior Data Engineer
Experience: 2-5 Years
Location: Melbourne, Australia
Work Mode: Remote
Job Type: Remote Contract / Full-Time
Job Summary
We are looking for a motivated Junior Data Engineer with hands-on experience or a strong foundational understanding of Databricks, Medallion Architecture, Azure/AWS cloud platforms, SQL, PySpark, and Python. The ideal candidate will support the development, maintenance, and optimization of data pipelines while working with senior data engineers to build scalable and reliable data solutions.
Key Responsibilities
- Develop and maintain data pipelines using Databricks, PySpark, Python, and SQL.
- Work with Medallion Architecture, including Bronze, Silver, and Gold data layers.
- Assist in building data ingestion, transformation, and processing workflows.
- Work with Azure or AWS cloud services to support data engineering solutions.
- Perform data cleansing, validation, transformation, and quality checks.
- Assist in implementing ETL/ELT processes and troubleshooting pipeline issues.
- Work with Delta Lake and structured data processing in Databricks.
- Support data pipeline monitoring, performance optimization, and debugging.
- Collaborate with senior data engineers, architects, and other technical teams.
- Follow best practices for coding, documentation, data quality, and security.
Required Skills and Qualifications
- Basic to intermediate hands-on experience with Databricks.
- Understanding of Medallion Architecture (Bronze, Silver, and Gold layers).
- Good knowledge of SQL and relational databases.
- Programming experience in Python and PySpark.
- Exposure to Azure or AWS cloud platforms.
- Understanding of ETL/ELT concepts and data transformation processes.
- Basic understanding of Apache Spark and Delta Lake.
- Strong analytical, problem-solving, and communication skills.
- Willingness to learn new tools and technologies and work collaboratively.
Preferred Qualifications
- Experience with Databricks Auto Loader.
- Knowledge of Delta Live Tables (DLT) or Lakeflow Declarative Pipelines.
- Familiarity with Unity Catalog and data governance.
- Exposure to Git, version control, and CI/CD practices.
- Databricks certification is an added advantage.
Ideal Candidate Profile
A junior-level candidate with a strong foundation in data engineering and practical exposure to Databricks, SQL, PySpark, Python, Medallion Architecture, and Azure/AWS cloud services. The candidate should be eager to learn and contribute to real-world data engineering projects.