PLATFORM and HPC DATA ENGINEER - VIRGINIA -URGENT
Job Number: 182
Job Category: GovTech
Job Title: PLATFORM and HPC DATA ENGINEER - VIRGINIA -URGENT
Job Type: Full-time
Clearance Level: TS/SCI CI Poly
Work Arrangement: On-site
Job Location: Herndon VA
Background
- Design and implement data management systems and architectures for HPC platforms, focusing on optimizing data flow, storage, and access in large-scale computing environments
- Oversee the configuration, maintenance, and optimization of distributed file systems (e.g., Lustre, IBM Spectrum Scale, NFS, GPFS) and storage solutions used in HPC environments to ensure efficient performance, scalability, and reliability
- Implement and manage metadata-driven systems for data labeling/tagging. This includes the development of strategies for classifying, indexing, and organizing datasets to enhance data discoverability, access control, and auditing
- Configure and maintain various storage appliances (e.g., NetApp, Dell EMC, HPE) and integrated storage solutions. Ensure that storage
- Implement security best practices for data access, protection, and management, ensuring compliance with government regulations and internal data governance policies. Configure encryption, access control, and secure data sharing method devices are optimized for performance, capacity, and availability within the HPC ecosystem
- Develop and maintain automation scripts (e.g., using Python, Bash, or Perl) to streamline storage configurations, data labeling/tagging, and system monitoring tasks. Automate processes related to data integration and HPC platform management
Requirements
- Bachelor’s degree in computer science, information technology, engineering, or a related field. A Master’s degree or higher
- 7+ years of experience in managing data infrastructure in HPC environments, with expertise in file systems, storage appliances, and data workflows
- Hands-on experience with distributed file systems, including Lustre, IBM Spectrum Scale (GPFS), NFS, and others commonly used in HPC setting
- Proven experience with storage appliance configuration (e.g., NetApp, Dell EMC, HPE, or similar systems), including performance tuning, capacity management, and reliability
- Familiarity with data access protocols like GridFTP, rsync, and NFS for large-scale data transfer
- nowledge of high-performance networking protocols (e.g., InfiniBand, RDMA) and their role in data transfer and storage optimization
Preferred
- Experience with containerization (Docker, Singularity) in an HPC context for data processing and application deployment
- Familiarity with high-performance computing (HPC) schedulers (e.g., SLURM, PBS, Torque) and their interaction with data storage systems
- Experience with cloud storage integration or hybrid cloud environments, with knowledge of cloud-native storage solutions (e.g., AWS S3, Ceph, OpenShift)
Share Job: