Translating Navigation Instructions in Natural Language to a High-Level Plan for Behavioral Robot Navigation

Xiaoxue Zang*, Ashwini Pokle*, Marynel Vázquez, Kevin Chen, Juan Carlos Niebles, Alvaro Soto, Silvio Savarese

---

Overview

We propose an end-to-end deep learning model for translating free-form natural language instructions to a high-level plan for behavioral robot navigation. We use attention models to connect information from both the user instructions and a topological representation of the environment. We evaluate our model's performance on a new dataset containing 11,051 pairs of navigation instructions. Our model significantly outperforms baseline approaches. Furthermore, our results suggest that it is possible to leverage the environment map as a relevant knowledge base to facilitate the translation of free-form navigational instruction.

---

Links

• Arxiv Paper (PDF, 14MB)
• Code (Github)
• Bibtex

If you find our project helpful, please consider citing us:

@inproceedings{zang2018behnav,
  title={Translating Navigation Instructions in Natural Language to a High-Level Plan for Behavioral Robot Navigation},
  author={Zang, Xiaoxue. and Pokle, Ashwini, and Vázquez, Marynel and Chen, Kevin. and 
	Niebles, Juan Carlos. and Soto, Alvaro and Savarese, Silvio}
  },
  booktitle={Conference on Empirical Methods in Natural Language Processing (EMNLP 2018)},
  year={2018}
}
        

Dataset

Dataset Downloads

• Train / Validation / Test Dataset (ZIP, 10 MB) [Download]

• Navigation maps (ZIP, 1.4GB) [Download]

If any errors or artifacts in the dataset are found, please report them to ashwini@cs.stanford.edu or xzang@cs.stanford.edu. Thank you!

---

Acknowledgements

The Toyota Research Institute (TRI) provided funds to assist with this research, but this paper solely reflects the opinions and conclusions of its authors and not TRI or any other Toyota entity. This work is also partially funded by Fondecyt grant 1181739, Conicyt, Chile. The authors would also like to thank Gabriel Sepúlveda for his assistance with parts of this project.

* indicates that both the authors contributed equally.