I am looking for one or more partners to start learning and doing Data Engineering lessons and projects. My time zone is in Toronto.

​

I’ve been applying for Data Engineer roles through LinkedIn, company career pages, and other job portals, but I’m wondering how effective direct applications actually are.

For those who have changed jobs recently, did you get interview calls by applying directly, or did most of your calls come through referrals?

I have around 4–5 years of experience in data engineering, mainly with Azure Data Factory, SQL, Databricks, and PySpark.

I’d like to know:

- Have you received interview calls through direct applications recently?

- Are referrals significantly more effective?

- Roughly how many applications did it take for you to get a callback?

- What strategy worked best for you?

Would appreciate honest experiences, especially from people working in India’s IT industry.

Can someone give a referral for a job at HP? Will dm u the job link

Hi, I quit my job without an offer. I want to get an offer by January. If your company is hiring or looking for a Data Engineer. Do let me know!
And any referrals would be appreciated!

Thanks in advance!

I am in 3rd year preparing for DE roles but rarely any internship. I have DS projects too but similar situation having hard time getting info.
For Projects working on a Bigger DE project will put it on resume when at stage to be able to present.
Any advice on what to study or where to apply anything at all will be helpful feeling stuck

Hi gang,

I have recently completed my data science masters from reputable university but i am stuck some where to transit into a professional careers.

I have two foundational certifications like Aws cloud practitioner and azure fundamentals and I am preparing for fabric associate certification for a week now and i have several projects around data engineering in open source unlike using paid ones like fabric.

So where do i need to focus? What should i learn more? Why am i stuck ? And learn and giving fabric associate certification is worth doing ? Or am i wasting time ? Have lot of things in mind …

And i am not a hard core vibe coder i know vibe coding but i dont do code like slopocolypse sort of coding i prefer doing a planning, brain stromming, documenting and implementing sort of person so yeah there is lot running but this is for the moment …

We are a small technical team focused on GPU kernels and production inference systems. We help AI startups, inference API teams, GPU providers, and companies running self-hosted models make inference faster, more reliable, and less expensive.

If your team is responding to slow generation or high GPU bills by adding more hardware, we can first determine whether the real constraint is kernels, memory movement, quantization, KV-cache layout, batching, scheduling, model loading, or the serving architecture.

We can help with:

• CUDA and Triton kernel development, fusion, profiling, and benchmarking
• PyTorch model-path optimization and numerical validation
• vLLM, SGLang, TensorRT-LLM, and custom serving systems
• quantization, VRAM reduction, KV-cache layout, continuous batching, and CUDA Graphs
• time to first token, inter-token latency, throughput, and tail-latency analysis
• scale-to-zero, cold-start behavior, readiness, cancellation, streaming, and failure recovery
• GPU utilization, cost-per-request analysis, and capacity planning
• reproducible before/after benchmarks with correctness gates and documented limitations

A typical engagement starts with a focused inference audit:

1. Define the model, framework, GPU, traffic shape, latency or throughput target, and cost constraint.
2. Profile the current path and identify the dominant bottlenecks.
3. Agree on the smallest changes worth implementing.
4. Deliver code, before/after measurements, correctness checks, and remaining risks.
5. Help integrate and validate the optimized path under realistic load.

Good fits include a slow or expensive inference endpoint, VRAM pressure preventing the desired batch size or context length, unstable serving under load, poor GPU utilization, costly cold starts, or a custom kernel/model path that needs profiling and validation.

For an initial conversation, send a Reddit chat or private message with:

• model and framework
• GPU type and VRAM
• current latency, throughput, utilization, or cost problem
• target workload and success metric
• deployment environment
• timeline and approximate budget range

We will tell you directly whether the workload appears optimizable and what evidence would be needed before making a performance or savings claim.

Hi everyone!

I’m exploring Data Engineer opportunities with 3+ years of experience in PySpark, SQL, Spark, and AWS.
If your company is hiring, I’d appreciate a referral.

Happy to share my resume via DM.

Hi guys,

I have cleared thoughtworks interview rounds in Data engineer domain/role.

Just wanted to check on the data engineer project availability in thoughtworks.

Can anyone please provide me some suggestions or share any other thoughts if any.

Hello,

I am data engineer with 6+ years of experience in this area. I have experience in designing and building scalable data platforms across AWS, Azure, GCP, and modern data lakehouse architectures. Skilled in both batch and real-time data processing, data modeling, and building data products for analytics and business reporting. Experienced in working with distributed teams, stakeholders, and implementing end-to-end data solutions including governance, CI/CD, and infrastructure as code.

I am looking for something additional, I am open for any kind of ideas, offers:)

Hi everyone.

I’m currently looking for my first full-time position as a Data Engineer, and I’d appreciate some honest feedback on my CV.

I have an engineering degree specializing in Data Engineering and AI, with experience working on data pipelines, data ingestion and transformation, Python microservices, data lakes, and RAG systems. I’ve also built a personal end-to-end batch and streaming data platform using Kafka, Spark, Airflow, PostgreSQL, and Grafana.

I’ve been applying for jobs, but I’m not getting as many responses as I expected, so I’d like to understand whether my CV might be part of the problem.

I’d really appreciate your feedback on a few points:

• Is my CV clear and well-structured?
• Does it position me well for junior Data Engineer roles?
• Are my professional experiences described effectively, or should I rewrite some bullet points?
• Are there any skills or technologies I should highlight more or remove?
• Is there anything that looks weak, unclear, exaggerated, or irrelevant?
• What would you change to make my CV more attractive to recruiters?

I’m looking for constructive criticism, so please feel free to be direct.

Thanks in advance to anyone willing to take the time to review it.

Hey everyone!

I'm looking for some genuine career advice from people who are already working in the IT industry, especially in Data Engineering, Data Science, Cloud, AI/ML, or related fields.

I'm a fresher, and lately, I've been getting quite stressed after hearing so many people say that AI is going to replace software and IT jobs. After doing some research, I came across Azure Data Engineering and found people saying that it could be a good career path for freshers.

However, I don't want to blindly follow a trend or spend months learning something without understanding its future scope.

A little about me:

• I have a B.Tech in Computer Science.
• I'm preparing for my UK master's: MSc Big Data and Data Science Technology with Advanced Practice at Northumbria University, London.
• I want to use my preparation time wisely and build skills that will actually help me get a job.
• I'm interested in Data Engineering, Data Science, AI/ML, and Cloud, but I'm still confused about which direction to choose.

I'd really appreciate your advice on a few questions:

1. Is Azure Data Engineering a good career choice for freshers in 2026? Is it realistic to get an entry-level job without prior professional experience?
2. With AI advancing so quickly, how safe is Data Engineering compared with Data Analytics, Data Science, Software Development, and Cloud Engineering?
3. Are there any IT fields that you believe will have good long-term demand and offer reasonable opportunities for freshers?
4. If you were starting your career from scratch today, which field would you choose, and why?
5. What skills should I focus on before starting my master's? For example, SQL, Python, Azure, Databricks, Spark, or something else?
6. Is it better to specialise in one field first or build a foundation in Data Engineering and then move towards AI/ML?
7. Since I'm pursuing my master's in the UK, which skills and career paths would give me better opportunities in both the UK and India?

I'm not expecting any field to be 100% safe from AI. I just want to make an informed decision, focus on valuable skills, and avoid wasting my limited preparation time jumping between different courses and career paths.

I'd especially appreciate advice from people who have actually worked in these fields or hired freshers. Please share your honest experiences, including the challenges, competition, and what the job market is really like.

Thanks in advance! Any practical guidance would genuinely help me a lot.

I work at a Big 4 in Belgium and have around 2.5 years of total experience. My role is mainly data analytics rather than data engineering. I do ETL transformations of financial data into dashboards using Alteryx Designer / SQL (sometimes Python but rarely). I have a Master’s in Economics with a major in Data Science (which inluded a data engineering course).

I want to move into an industry data engineering role, but I realise my experience is not the same as what is expected of a dedicated data engineer. I’m totally fine starting as a junior, because realistically that is what I am. During my graduation year, I applied for data roles but never got far in the process, mostly because they were looking for more purely technical academic backgrounds. Now that I have hands-on data analytics and ETL experience, I feel it may be worth trying again.

I’m aiming to start applying next year and am currently sharpening my skills and building a portfolio. How realistic is it to get a Data Engineer role in Belgium in 2027 with the advance of AI? Would you count my ETL work as relevant experience? What do you think I could leverage in interviews, and what would be the main stopper for someone with my background?

About Reddit:

Reddit is a community of communities. It’s built on shared interests, passion, and trust, and is home to the most open and authentic conversations on the internet. Every day, Reddit users submit, vote, and comment on the topics they care most about. With 100,000+ active communities and approximately 130 million daily active unique visitors, Reddit is one of the internet’s largest sources of information. For more information, visit www.redditinc.com.

Who We Are:

The Infrastructure organization enables Reddit to deliver reliability, performance, and efficiency with a single opinionated technology stack. The Ingestion Platform team within Infrastructure builds a low‑code interface for moving data from supported sources to supported sinks, making it easier for teams across Reddit to create reliable, scalable data pipelines without building and operating bespoke infrastructure. Our platform supports streaming and batch data movement from Kafka to destinations such as BigQuery and object storage, and is expanding toward S3/GCS, Apache Iceberg, transformations, deduplication, and additional sinks. We work closely with Data Pipelines, Data Movement, Ads Data Platform, ML Indexing, Flink Platform, Storage, and other partner teams to improve developer velocity, reliability, observability, and operational efficiency.

We are looking to hire a Staff Software Engineer to help shape and scale this platform. This is a high‑impact role where your technical judgment, customer empathy, and operational mindset will influence the future direction of data movement at Reddit.

What You’ll Do:

• Design, write, and deliver reliable software for distributed data movement across streaming and batch workloads, with a focus on availability, scalability, latency, correctness, and cost efficiency.
• Own the architecture and evolution of the platform’s control plane and data plane, including pipeline APIs, controllers, connectors, target managers, schemas, and sink integrations.
• Expand the platform beyond Kafka‑to‑BigQuery by helping deliver production‑ready paths to S3/GCS and Apache Iceberg, as well as transformations, deduplication, dead‑letter queues, and other reusable capabilities.
• Build high‑quality connectors and abstractions for systems such as Kafka, BigQuery, S3, GCS, Iceberg, Flink, and other data stores, while keeping the platform modular and easy to extend.
• Improve the self‑service experience for platform users through clear APIs, safe defaults, automated provisioning, documentation, onboarding workflows, and actionable observability and alerting.
• Lead migrations from bespoke and legacy ingestion systems to the Ingestion Platform, partnering with teams such as Ads Data Platform and ML Indexing to make migrations safe, incremental, and low‑touch.
• Establish strong reliability, security, and operational practices for pipelines running across Kubernetes clusters, including schema evolution, workload identity, permissions, deployment safety, monitoring, and incident response.
• Identify gaps in the platform’s architecture and product experience, and lead redesigns that improve developer velocity and enable Reddit’s continued growth.
• Work collaboratively with engineers and stakeholders across Infrastructure, Data Platform, Product, Ads, ML, Storage, and partner teams to align roadmaps and deliver durable solutions.
• Mentor and guide backend and data infrastructure engineers across the company, raising the bar for technical design, operational excellence, and customer focus.

Who You Might Be:

• 10+ years of hands‑on experience building internet‑scale distributed systems, data infrastructure, or platforms used by other developers.
• BS, MS, PhD in Computer Science, a related field, or equivalent practical experience.
• Strong software development experience in one or more general‑purpose languages such as Go, Python, Java, or Scala.
• Deep experience designing and operating high‑throughput, fault‑tolerant data pipelines or platform services, including streaming and batch processing.
• Experience with data movement patterns and systems such as Kafka, BigQuery, S3, GCS, Apache Iceberg, Flink, or comparable technologies.
• Experience designing APIs and platform abstractions, including schema evolution, serialization formats such as Protocol Buffers, and service‑to‑service communication such as gRPC.
• Familiarity with Kubernetes and cloud‑native platform concepts, including controllers, custom resources, workload identity, infrastructure automation, and multi‑cluster deployments.
• A strong understanding of observability and operational excellence: metrics, logging, alerting, SLOs, incident response, data quality, and safe migration practices.
• A product‑minded engineer who advocates for platform users and can turn ambiguous customer and partner needs into simple, reliable, and scalable solutions.
• Demonstrated technical leadership and a history of guiding teams through complex, cross‑functional initiatives with meaningful impact.
• Exceptional written and verbal communication skills, with the ability to collaborate effectively across engineering teams and stakeholders.
• Experience building low‑code or self‑service developer platforms, or curiosity to learn, is a strong plus.

Benefits

• Comprehensive Healthcare Benefits and Income Replacement Programs
• 401k with Employer Match
• Global Benefit programs that fit your lifestyle, from workspace to professional development to caregiving support
• Family Planning Support
• Gender‑Affirming Care
• Mental Health & Coaching Benefits
• Flexible Vacation & Paid Volunteer Time Off
• Generous Paid Parental Leave

Pay Transparency:

This job posting may span more than one career level.

In addition to base salary, this job is eligible to receive equity in the form of restricted stock units, and depending on the position offered, it may also be eligible to receive a commission. Additionally, Reddit offers a wide range of benefits to U.S.-based employees, including medical, dental, and vision insurance, 401(k) program with employer match, generous time off for vacation, and parental leave.

To provide greater transparency to candidates, we share base salary ranges for all US-based job postings regardless of state. We set standard base pay ranges for all roles based on function, level, and country location, benchmarked against similar stage growth companies. Final offer amounts are determined by multiple factors including, skills, depth of work experience and relevant licenses/credentials, and may vary from the amounts listed below.

The base salary range for this position is:

$217,000 — $303,900 USD

In select roles and locations, the interviews will be recorded, transcribed and summarized by artificial intelligence (AI). You will have the opportunity to opt out of recording, transcription and summarization prior to any scheduled interviews.

During the interview, we will collect the following categories of personal information: Identifiers, Professional and Employment‑Related Information, Sensory Information (audio/video recording), and any other categories of personal information you choose to share with us. We will use this information to evaluate your application for employment or an independent contractor role, as applicable. We will not sell your personal information or disclose it to any third party for their marketing purposes. We will delete any recording of your interview promptly after making a hiring decision. For more information about how we will handle your personal information, including our retention of it, please refer to the Candidate Privacy Policy for Potential Employees and Contractors.

Location: Remote - United States

Apply: Staff Software Engineer - Ingestion Platform at Reddit

Hi All,

My name is Zafar Waris and I am actively looking for Data Engineer opprtunities. I have 3 years of experience and my current tech stack includes (not limited to) Python, SQL, PySpark, Databricks, Snowflake, AWS, Airflow, Terraform.

If you have any openings in your org and come to know about any in your network, please help a brother out and DM for the same.

I will be waiting for any responses.

In the meantime, if anyone here would like to discuss anything on the yech stack I mentioned or have any questions around them, do not hesitate to reach out, I will try the best of my capacity to provide a satisfactory answer.

Thanks. LinkedIn - zmwaris1/Zafar Mahmood Waris

I'm Data engineer with more than 10 years of exp.

This opening is in my team.

Skills

Mandatory skills: Python, sql, spark, Airflow, Aws glue, Apache Iceberg,

preferred but not mandatory: Kafka, cdc

Please DM me only if you have these ticks

Tick1 : all mandatory skills

Tick2 : 6 plus years of exp.

Tick3 : can join within 30 days.

Tick4 : can work 2 days hybrid hinjewadi ph1.

What we have: Chill project, EU client, No one asks if you couldn't work much due to some issue. No micromanagement, Supporting team, guidance wherever required.

​

Hi everyone!

I'm a Data Engineer with an engineering degree in Data Engineering and professional experience building data pipelines and analytics solutions. I'm currently looking for freelance missions alongside my main job, preferably remote projects with international clients.

My technical stack

Cloud & Data Engineering: Azure, Azure Data Factory (ADF), Azure Databricks, ADLS Gen2

Processing: Python, PySpark, SQL, Delta Lake

Architecture: ETL/ELT, Medallion Architecture (Bronze/Silver/Gold), Data Warehousing, Dimensional Modelling

Analytics & BI: Power BI, analytical datasets, fact and dimension tables

DevOps: Azure DevOps, Git, CI/CD for data pipelines and Databricks deployments

I'm also certified as a Databricks Certified Data Engineer Professional, Microsoft Certified: Azure Data Engineer Associate, and Microsoft Certified: Fabric Data Engineer Associate.

Examples of problems I can help solve

1. Building an end-to-end data pipeline

Imagine a company receiving data from multiple sources that needs to be consolidated into a reliable analytics platform. I can help build an Azure Data Factory ingestion workflow, process the data with Databricks and PySpark, organize it into Bronze/Silver/Gold layers using Delta Lake, and prepare curated datasets for reporting.

2. Fixing unreliable or inefficient ETL processes

For pipelines struggling with duplicates, historical records or repeated full reloads, I can help implement incremental processing, deduplication, validation checks and more maintainable transformation logic.

3. Designing a data warehouse for reporting

I can transform raw or processed datasets into fact and dimension tables using star-schema modelling, making business data easier to query and consume through Power BI.

4. Automating data engineering deployments

I can help structure development, staging and production environments, package reusable Python code, and automate deployments of Databricks jobs and other resources through Azure DevOps CI/CD pipelines.

5. Preparing data for business intelligence

I can help integrate and clean data from different sources, improve analytical data models, and prepare reliable datasets for dashboards and business reporting.

These examples reflect the kinds of solutions I've worked on professionally and can help deliver. I'm strongest in Azure, Databricks, SQL and data pipeline development.

What I'm looking for

Short-term freelance missions or long-term collaborations

Azure Data Factory and Databricks projects

PySpark, SQL and ETL/ELT development

Data quality, data integration and data modelling

Data warehousing and Power BI-related projects

I'm based in Morocco and available for remote collaboration, mainly during evenings, with some flexibility.

If you need help with a data engineering project, feel free to DM me with a short description of your requirements. I'd be happy to discuss how I could contribute.

Thanks for reading!

Hi everyone,

I’m currently looking for new opportunities as a Data Engineer, Fivetran/HVR Developer, or Databricks Data Engineer in India.

I have 3+ years of experience in Data Engineering and currently work at Genpact, supporting GE Vernova as a client.

My key skills and experience include:

• Fivetran HVR: HVR 5/6 administration, real-time data replication, capture jobs, source configuration, supplemental logging, monitoring, and production troubleshooting.
• Databricks: Hands-on exposure to data integration and replication workflows.
• SQL: Querying, troubleshooting, and data validation.
• Database Replication: PostgreSQL, SAP sources, and integration with platforms such as Snowflake, Redshift, and S3.
• Automation: Shell scripting, monitoring, alerting, and production support.

I’m open to relevant opportunities across India, including Gurugram, Noida, Bangalore, Hyderabad, Pune, and other locations.

If you know of any suitable openings or can refer me to a relevant team, I’d really appreciate your help. I’m happy to share my resume.

Thank you!

Hi All,

My name is Zafar Waris and I am actively looking for Data Engineer opprtunities. I have 3 years of experience and my current tech stack includes (not limited to) Python, SQL, PySpark, Databricks, Snowflake, AWS, Airflow, Terraform.

If you have any openings in your org and come to know about any in your network, please help a brother out and DM for the same.

I will be waiting for any responses.

In the meantime, if anyone here would like to discuss anything on the yech stack I mentioned or have any questions around them, do not hesitate to reach out, I will try the best of my capacity to provide a satisfactory answer.

Thanks. LinkedIn - zmwaris1/Zafar Mahmood Waris

Hello All,

My name is Aakash. I have 3 yrs of exp as a Data Engineer . I have been actively looking jobs related to data engineer roles.

Tech stack :- Databricks, Pyspark, Hadoop, Hive, Data Modelling,

Cloud:- Azure, GCP, Azure

LinkedIn:- aakashkumardetne (Dm me I will share the link)

If you have any openings or if you can refer me, Please dm . I am an Immediate joiner

Hi y'all I wish success for all of you, any of you in case know abou the pvr cloud tech institute in Hyderabad teaching azure data engineering and also started teaching full stack azure data engineering?

https://www.linkedin.com/safety/go/?url=pvrcloudtech%2Ecom&urlhash=odOT&mt=0iyOWH4fY0IOBq03r7kIjazSHLzpJaNny44HAF00alDrxuTREFbuHZq3A0GfRBh_q8GnDWVY0EFct_8GEv7UwT6UD6kS&isSdui=true

Hello,

I am a looking out for part-time freelance opportunities as a
Data Engineer.
I have 6 years of Experience as a Data Engineer. I have worked on streaming as well as batch use cases.
I have architectured multiple end to end pipelines. I have experience in Azure Services,Airflow , Snowflake, Pyspark,
SQL.
Please let me know if there are any opportunities for me.

Hi, I'm a Snowflake developer(I was doing day to day BI,but am SnowPro Core Certifified) in big4 with 4YOE.Im looking for a mentor who can guide me to getting an offer from Product Based MNC. More than happy to pay for your time,Please dm if interested

Kindly dm if you have any experience