With advancements in robotics and automation, technology is moving toward transforming mine fleets into autonomous robots. While mine fleets engaged in load and haul operations typically follow defined routes, other equipment, such as utility vehicles and robots used for inspection, search, and rescue, will need to operate in undefined environments. This paper explores the application of Large Language Models (LLMs) for autonomous operations in both defined and undefined underground mining environments. LLMs are finding new frontiers in promoting the autonomy of robots, understanding high-level commands, and navigating robots in a way that adheres to the many variations in the environment. This study presents LLM-NavAgent, an Al agent that harnesses the power of LLMs for ground robot navigation with a focus on mining. The mining environment is characterised by moving objects, changing surface levels, and little or no light, making it extremely difficult, thus demanding a high level of adaptability and quick decision-making. With the advancements in LLM-based navigation such as LM-Nav's use of language, vision, and action for autonomous instruction following, also integrated with dynamic navigation in cluttered environments with SayNav, LLM-NavAgent adapts these approaches to address the challenges posed by mining environments. The agent merges LLM-derived natural language processing and ground robot control systems to facilitate the successful execution of high-level commands such as navigation in congested mining environments, obstacle avoidance, and mission completion in a short period of time. This research applies LLM-NavAgent to the minimum viable product in mining operations and addresses scenarios that include performing inspections, face mapping and autonomous exploration. Accordingly, this work contributes to the comprehension of LLM usage in mining robotics, and it conceptually establishes that the LLM-NavAgent represents a step toward adaptable and intuitive robotic navigation in the underground mining environment.