Splunk Engineer

Job Description

Job Description

Pueo is known for bringing the best talent and unique tools to every opportunity. Pueo's Parliament (aka workforce) is composed of professionals who are seeking the opportunity to work in a business organization that thrives on career development and independence. In support of mission and professional growth, our Parliament has supported the development of multiple patents, proprietary tools, and applications as well as trademarked processes.

Our organization emphasizes career development across multiple career environments (at the members own pace) and ensures those who contribute broadly are properly rewarded. Pueo has four career environments where every member of the parliament can participate. Each environment has opportunities available for all levels. Opportunities are framed by an employee's desires and capabilities, and we ensure challenges, growth, and unique experiences are available for employees at all levels.

Our Career Environments (Program, Functional, Service, and Leadership) provide numerous opportunities for employees to invest in their personal growth and those things that offer fulfillment. We invest in helping our members create and execute their career development plans. Our Pods (small teams of 5 or less) are comprised of personnel with similar skillsets to ensure mentorship, understanding, and peer support.

OVERVIEW:

The IA&E Engineer provides advanced cybersecurity engineering and infrastructure expertise in support of Treasury's enterprise-wide SOC operations under TESIEMS. This position is responsible for designing, integrating, and maintaining secure compute, storage, and virtualization environments across both on-premises data centers and cloud environments (AWS). Additionally, this role configures and troubleshoots Splunk platform components, focusing on resolving connectivity and performance issues across the Splunk environment. Ensure platform availability, data integrity, and reliability of log flow from multiple sources. The engineer applies deep technical knowledge of Linux systems, virtual machines, and enterprise infrastructure to optimize SOC monitoring, strengthen Treasury's cybersecurity posture, and ensure resilient, scalable operations across the Department.

GENERAL DUTIES:

• Primary responsibilities include design, configure, and maintain low-level Splunk platform components including forwarders, indexers, search heads, and deployment servers.
• Troubleshoot system outages, downtimes, and performance degradation.
• Diagnose and resolve issues with unresponsive or misconfigured hosts and forwarders. Investigate and remediate syslog ingestion issues, including network connectivity, parsing, and logging issues.
• Support integration and troubleshooting of security and email gateway logs, including IronPort data sources.
• Configure, maintain, and optimize Splunk configuration files including props.conf, inputs.conf, transforms.conf, and related configurations.
• Analyze logs to identify root causes of ingestion delays, dropped events, or parsing errors.
• Monitor data pipelines to ensure consistent log ingestion, normalization, and indexing.
• Collaborate with system administrators, network teams, and security teams to resolve cross-platform issues.
• Document troubleshooting procedures, configuration changes, and operational best practices.
• Develop dashboards and Splunk Health Checks to check intermediate forwarder connectivity, search head availability, disk utilization, and license usage.
• Configure and manage server classes to control forwarder configurations, app distribution, and deployment across the environment.
• Manage and troubleshoot Cluster Manager controls, including index replication, bucket fixing, peer status, and cluster health.
• Configure, deploy, and support intermediate forwarders, including load balancing, filtering, routing, and data normalization.
• Coordinate planned maintenance, upgrades, and configuration changes ensuring minimal disruption to data ingestion and search availability.
• Develop and maintain operational runbooks, health check procedures, and escalation workflows.

REQUIRED QUALIFICATIONS:

• Seven (7) years of experience. Three additional years of experience in lieu of degree.
• Deep Splunk technical knowledge and proficiency.
• Problem-solving complex Splunk data flow and configuration technical issues.
• Proven experience supporting Splunk in a production environment with a focus on low-level operations and troubleshooting.
• Strong understanding of Splunk data ingestion pipelines, forwarders, indexing processes, and search performance.
• Hands-on experience configuring and troubleshooting props.conf, inputs.conf, and transforms.conf.
• Strong Linux/Unix system administration skills, including log analysis, process monitoring, and network troubleshooting.
• Excellent problem-solving skills and strong communication skills for coordinating with technical teams.
• Strong analytical and critical thinking skills.
• Understanding of impact assessment on end-products or solutions.
• Broad technical understanding of related cybersecurity specialty areas.
• Ability to develop and implement technical solutions independently.
• Familiarity with incident detection, response, and security event management. Proficiency in tools such as SIEMs (e.g., Splunk), IDS/IPS, endpoint detection, and scripting languages.
• Familiarity with NIST SP 800-53, FISMA, and risk management frameworks.
• Experience with scripting (e.g., Python, Bash) and log data analysis.

DESIRED QUALIFICATIONS:

• Bachelor's degree from an accredited institute in an area applicable to the position in Cybersecurity, Computer Science, Information Systems, or a related discipline.
• Relevant certifications, such as Certified Information Systems Security Professional (CISSP), Certified Information Security Manager (CISM), Cyber Security Analyst + (CySA+) are highly desirable.

CLEARANCE:

• Secret minimum

Pueo is an equal employment opportunity employer and affirmative action employer. All interested individuals will receive consideration and will not be discriminated against on the basis of race, color, religion, sex, national origin, disability, age, sexual orientation, gender identity, genetic information, or protected veteran status. Pueo takes affirmative action in support of its policy to advance diversity and inclusion of individuals who are minorities, women, protected veterans, and individuals with disabilities.

Job Posted by ApplicantPro