SV
← All work
002 Case study · Robotics · RL · ROS
⟁

Hotel Delivery Robot

Learns to deliver. Doesn't follow a script.

An autonomous robotic delivery system for hotel environments. A differential-drive robot navigates a 22-room floor plan, and learns — through 30,000 epochs of simulation — which delivery sequence maximises on-time performance, including dynamic priority for urgent rooms.

The work

Overview

The naive solution — visit rooms in a fixed order or shortest-distance sequence — ignores real-world factors: travel time variance, room priority, and the compound cost of late deliveries to high-priority guests. This project explores whether a reinforcement learning agent, trained on actual simulated navigation times, can learn a better policy than any hand-coded heuristic. The robot was designed from scratch in Fusion 360, converted to URDF via fusion2urdf, and operates in a bespoke 22-room Gazebo world. The 2D occupancy map is generated via SLAM (gmapping); at runtime the robot localises using AMCL and executes navigation goals via the ROS move_base action client.

Architecture

How it's built

SIMGazebo22-room URDFMAPSLAM(gmapping)LOCALISEAMCLDECIDESARSAQ-learningNAVIGATEmove_base

Fig. 002.1 · From a SLAM-built map the robot localises with AMCL; the Q-learning agent picks the next room and move_base drives there, closing the loop back into the simulation.

Detail

Key features

  • 22-room hotel world in Gazebo; robot designed in Fusion 360 and simulated via URDF with custom mesh files
  • SLAM (gmapping) for map generation, AMCL for runtime localisation, move_base for autonomous navigation goal execution
  • SARSA-style Q-learning with epsilon-greedy exploration (α = 0.5, γ = 0.8), trained over 30,000 simulation epochs on real navigation times
  • Reward shaping that teaches the agent to prioritise 6 designated rooms (+20 on-time vs. +10 normal) without explicit sequencing rules — behaviour emerges from training
  • 22×22 Q-value and travel-time matrices persisted as CSV between sessions; agent retains learned policy across runs without retraining
  • Four-module software architecture: state manager, Q-learning worker, navigation interface (move_base wrapper), and main delivery controller
  • Runtime Trainer / Tester mode switch — keep training the policy or replay a learned one, no code changes

Imagery

In context

Fig. 002.2 · Gazebo 22-room world
Fig. 002.3 · Learned delivery path

Results

Impact

  • Agent learns to service priority rooms first — without being programmed to — purely from reward shaping over 30,000 epochs
  • Q-table encodes experiential knowledge from a physics-simulated environment with real sensor noise, map uncertainty, and path-planning variance — not a static distance matrix
  • Clean separation of concerns: navigation (ROS/AMCL), decision-making (Q-learning), and environment state are fully decoupled — structure extends naturally to multi-robot or multi-floor scenarios
  • Fully functional in ROS Noetic + Gazebo; demo video and source code available on GitHub

Open source

How to contribute

The stack is fully decoupled — navigation, decision-making, and environment state are separate modules. Good places to start:

  • Clone the repo and run the 22-room Gazebo world in ROS Noetic
  • Swap the SARSA reward shaping to experiment with new priority schemes
  • Extend the navigation interface toward multi-robot or multi-floor delivery
  • Open an issue with a delivery scenario you'd like the agent to learn
← PreviousStrata

Let's build the next one together.

If this is the shape of work you're trying to ship, I'd love to hear about it. Bring me the question — the answer can be co-authored.

Start a conversation →