Visual Sign Parsing & 3D Grounding

Navigational Sign Taxonomy

A research-oriented taxonomy organized by where signs appear, what they encode, and how they update navigation state.

Sign → information type → structured information → navigation-state update

1. Navigation information flow

Different sign types support different stages of a journey.

Where is my goal?

Goal localization

DirectoryFloor plan

Which way now?

Route decision

DirectionalFingerpost

How do I change level?

Transition

LiftStairsEscalator

Still right / arrived?

Confirmation

ReassuranceIdentification

Constraint / Dynamic signs can modify any stage: No Entry, Lift Out of Service, Detour, temporary closures, etc.

2. Real-world visual context

Representative contexts for routing, vertical transition, and map-based localization.

Overhead routing
Destination + arrow + pictogram.

Vertical transition
Lift / stairs / local directions.

Directory / floor plan
Global layout and goal localization.

3. Seven navigation-relevant classes

Each class has a different natural output and a different 3D grounding problem.

Directional Sign

corridor · junction · airport · mall · hospital

Visual pattern

Destination + arrow + optional pictogram; often several destination–direction pairs.

Function

Choose the next local branch.

State update

destination → candidate path / bearing

3D challenge: Map a 2D arrow to the correct traversable corridor in 3D, especially with overlapping branches or oblique sign views.

Vertical-transition Sign

lift lobby · stairs · escalator · multi-level hub

Airport sign directing travelers to elevator and escalators/stairs. Photo: Bri · Wikimedia Commons

Visual pattern

Floor number/range + arrows + lift/stairs/escalator icons.

Function

Select a vertical connector and target level.

State update

floor_i —connector→ floor_j

3D challenge: Disambiguate whether ↑ means straight ahead, upstairs, or up escalator; reason over explicit multi-floor topology and Δz.

Map / Floor Plan

mall · hospital · campus · museum · station

Floor map with “You are here”, stairs, lifts and destinations. Photo: Ciara Ní Riain · Wikimedia Commons

Visual pattern

Schematic geometry + labels + legend + You Are Here.

Function

Localization + global route planning.

State update

2D schematic → topological / metric graph

3D challenge: Align a schematic map to the robot/world frame and recover vertical connectors across floors.

Outdoor Fingerpost / Pylon

campus · park · plaza · trail

Outdoor fingerposts at Jurong Bird Park, Singapore. Photo: Marcin Konsek · Wikimedia Commons

Visual pattern

Multiple physical panels or one pylon, often with distance.

Function

Choose an outdoor heading/path.

State update

destination → world-space heading

3D challenge: Physical panel orientation may carry the direction; cropping the sign can remove the decisive 3D cue.

Identification / Reassurance

along route · floor landing · destination entrance

Destination identification sign mounted at the room entrance. Photo: Beyond My Ken · Wikimedia Commons

Visual pattern

Destination name, room/floor number, or continue-straight cue.

Function

Confirm route state or goal arrival.

State update

belief(route/goal) ↑

3D challenge: Associate a sign with the correct physical room/building/floor and distinguish confirmation from a new action.

Constraint / Dynamic Sign

temporary closure · staff-only · detour · live update

A physical “No Entry” sign constraining traversability. Photo: Pivan0009 · Wikimedia Commons

Visual pattern

No-entry/restricted icons, temporary boards, printed notices, digital messages.

Function

Modify traversability or override stale prior knowledge.

State update

edge available → blocked / redirected

3D challenge: Ground a restriction to the correct door/path/connector and update the graph temporally.

4. Research matrix

How each sign type maps into a robot’s internal navigation representation.

Class	State update	Representation	Key 3D challenge
Directional	Next branch	Path / bearing	2D arrow → 3D path
Vertical transition	Floor / connector	Multi-floor graph edge	↑ semantics + Δz
Directory	Goal location	Destination → floor/zone	symbolic floor → topology
Map / floor plan	Localization + route	Topological / metric map	schematic → world frame
Fingerpost	Outdoor heading	World-space direction	sign pose → heading
Identification / reassurance	Belief / termination	Route-state confidence	sign → physical entity
Constraint / dynamic	Traversability	Graph edge availability	restriction → edge + time

Project implication. A realistic benchmark should first classify what navigation information a sign provides, then evaluate parsing and grounding with a task-specific target. Treating every sign as “destination + arrow” misses directories, maps, vertical topology, confirmation, and dynamic constraints.