This is a remote position.
Job Title: Databricks Developer (Java & Apache Spark) Location: RemoteDuration: Full-TimeNEED IRS MBI Clearance.We are looking for a Databricks Developer who can design, build, and support large-scale data processing solutions. This role focuses on working with complex, high-volume data in a secure environment. You should be comfortable writing clean Java code, building Spark jobs, and improving performance in distributed systems.You’ll work closely with engineers, analysts, and business teams to deliver reliable data pipelines that scale well and meet security and compliance requirements.What You’ll DoBuild and maintain data pipelines using Apache Spark on DatabricksWrite clean and efficient Java (Java 8 or higher) following best coding practicesWork with large structured and semi-structured data setsTune Spark jobs to improve performance, stability, and costPartner with other teams to understand requirements and deliver solutionsMake sure data handling follows security and governance standardsInvestigate and fix issues in production environmentsWhat We’re Looking ForActive federal background clearance (client-issued laptop is a plus)Bachelor’s degree in Computer Science or a related field8+ years of hands-on experience in data engineering or big data developmentTechnical SkillsData & SystemsExperience working with large enterprise or government data systemsGood understanding of data structures, security, and complianceJavaStrong experience with Java 8 or newerComfortable with Streams, Lambdas, and object-oriented designApache SparkHands-on experience with Spark Core, Spark SQL, and DataFramesUnderstanding of RDDs and when to use themExperience with batch processing and streamingStrong skills in Spark performance tuningAble to debug jobs using Spark UI and logsBig Data ToolsExperience with tools like HDFS, Hive, or HBaseFamiliar with Kafka, S3, or cloud-based data lakesExperience with Parquet, Avro, or ORC formatsETL & Data ProcessingExperience building batch and real-time ETL pipelinesSkilled in data cleansing, transformation, and enrichmentDevOps & DeploymentExperience running Spark on YARN, Kubernetes, or cloud platformsFamiliar with CI/CD tools like Jenkins or GitHub ActionsExperience with monitoring tools and production logsTesting & Code QualityUnit testing using JUnit or TestNGExperience with Mockito or similar toolsComfortable validating data and testing Spark jobsWays You’ll SucceedYou’re used to working in Agile teamsYou document your work clearlyYou stay calm and focused when troubleshooting production issuesNice to Have:Experience with Scala or Python in SparkHands-on experience with Databricks or Google DataprocKnowledge of Delta Lake or Apache IcebergBackground in data modeling and performance design
3M Consultancy is a prime contractor specializing in federal government modernization and data engineering solutions, with particular expertise in IRS programs and initiatives. The company develops large-scale data processing solutions, ETL/ELT pipelines, and advanced analytics platforms using technologies like Databricks, Palantir Gotham, and Palantir Foundry. They focus on high-impact projects involving financial data integration, tax compliance, fraud detection, and risk assessment models for federal agencies. 3M Consultancy combines deep technical expertise in big data engineering with specialized knowledge of federal security requirements and compliance standards, delivering mission-critical solutions to government clients.
