Evaluating Indoor Positioning Systems in a Shopping Mall: The Lessons Learned from the IPIN 2018 Competition

Valerie Renaudin; Miguel Ortiz; Johan Perul; Joaquin Torres-Sospedra; Antonio Ramon Jimenez; Antoni Perez-Navarro; German Martin Mendoza-Silva; Fernando Seco; Yael Landau; Revital Marbel; Boaz Ben-Moshe; Xingyu Zheng; Feng Ye; Jian Kuang; Yu Li; Xiaoji Niu; Vlad Landa; Shlomi Hacohen; Nir Shvalb; Chuanhua Lu; Hideaki Uchiyama; Diego Thomas; Atsushi Shimada; Rin Ichiro Taniguchi; Zhenxing Ding; Feng Xu; Nikolai Kronenwett; Blagovest Vladimirov; Soyeon Lee; Eunyoung Cho; Sungwoo Jun; Changeun Lee; Sangjoon Park; Yonghyun Lee; Jehyeok Rew; Changjun Park; Hyeongyo Jeong; Jaeseung Han; Keumryeol Lee; Wenchao Zhang; Xianghong Li; Dongyan Wei; Ying Zhang; So Young Park; Chan Gook Park; Stefan Knauth; Georgios Pipelidis; Nikolaos Tsiamitros; Tomas Lungenstrass; Juan Pablo Morales; Jens Trogh; David Plets; Miroslav Opiela; Shih Hau Fang; Yu Tsao; Ying Ren Chien; Shi Shen Yang; Shih Jyun Ye; Muhammad Usman Ali; Soojung Hur; Yongwan Park

doi:10.1109/ACCESS.2019.2944389

Evaluating Indoor Positioning Systems in a Shopping Mall: The Lessons Learned from the IPIN 2018 Competition

Valerie Renaudin, Miguel Ortiz, Johan Perul, Joaquin Torres-Sospedra, Antonio Ramon Jimenez, Antoni Perez-Navarro, German Martin Mendoza-Silva, Fernando Seco, Yael Landau, Revital Marbel, Boaz Ben-Moshe, Xingyu Zheng, Feng Ye, Jian Kuang, Yu Li, Xiaoji Niu, Vlad Landa, Shlomi Hacohen, Nir Shvalb, Chuanhua LuHideaki Uchiyama, Diego Thomas, Atsushi Shimada, Rin Ichiro Taniguchi, Zhenxing Ding, Feng Xu, Nikolai Kronenwett, Blagovest Vladimirov, Soyeon Lee, Eunyoung Cho, Sungwoo Jun, Changeun Lee, Sangjoon Park, Yonghyun Lee, Jehyeok Rew, Changjun Park, Hyeongyo Jeong, Jaeseung Han, Keumryeol Lee, Wenchao Zhang, Xianghong Li, Dongyan Wei, Ying Zhang, So Young Park, Chan Gook Park, Stefan Knauth, Georgios Pipelidis, Nikolaos Tsiamitros, Tomas Lungenstrass, Juan Pablo Morales, Jens Trogh, David Plets, Miroslav Opiela, Shih Hau Fang, Yu Tsao, Ying Ren Chien, Shi Shen Yang, Shih Jyun Ye, Muhammad Usman Ali, Soojung Hur, Yongwan Park

Research output: Contribution to journal › Article › peer-review

73 Scopus citations

Abstract

The Indoor Positioning and Indoor Navigation (IPIN) conference holds an annual competition in which indoor localization systems from different research groups worldwide are evaluated empirically. The objective of this competition is to establish a systematic evaluation methodology with rigorous metrics both for real-Time (on-site) and post-processing (off-site) situations, in a realistic environment unfamiliar to the prototype developers. For the IPIN 2018 conference, this competition was held on September 22nd, 2018, in Atlantis, a large shopping mall in Nantes (France). Four competition tracks (two on-site and two off-site) were designed. They consisted of several 1 km routes traversing several floors of the mall. Along these paths, 180 points were topographically surveyed with a 10 cm accuracy, to serve as ground truth landmarks, combining theodolite measurements, differential global navigation satellite system (GNSS) and 3D scanner systems. 34 teams effectively competed. The accuracy score corresponds to the third quartile (75^th percentile) of an error metric that combines the horizontal positioning error and the floor detection. The best results for the on-site tracks showed an accuracy score of 11.70 m (Track 1) and 5.50 m (Track 2), while the best results for the off-site tracks showed an accuracy score of 0.90 m (Track 3) and 1.30 m (Track 4). These results showed that it is possible to obtain high accuracy indoor positioning solutions in large, realistic environments using wearable light-weight sensors without deploying any beacon. This paper describes the organization work of the tracks, analyzes the methodology used to quantify the results, reviews the lessons learned from the competition and discusses its future.

Original language	English
Article number	8852722
Pages (from-to)	148594-148628
Number of pages	35
Journal	IEEE Access
Volume	7
DOIs	https://doi.org/10.1109/ACCESS.2019.2944389
State	Published - 2019

Keywords

IPIN
competition
indoor positioning
smartphone

Access to Document

10.1109/ACCESS.2019.2944389

Cite this

Renaudin, V., Ortiz, M., Perul, J., Torres-Sospedra, J., Jimenez, A. R., Perez-Navarro, A., Martin Mendoza-Silva, G., Seco, F., Landau, Y., Marbel, R., Ben-Moshe, B., Zheng, X., Ye, F., Kuang, J., Li, Y., Niu, X., Landa, V., Hacohen, S., Shvalb, N., ... Park, Y. (2019). Evaluating Indoor Positioning Systems in a Shopping Mall: The Lessons Learned from the IPIN 2018 Competition. IEEE Access, 7, 148594-148628. Article 8852722. https://doi.org/10.1109/ACCESS.2019.2944389

@article{0338d06df07f4cb68918d2e94ae472d0,

title = "Evaluating Indoor Positioning Systems in a Shopping Mall: The Lessons Learned from the IPIN 2018 Competition",

abstract = "The Indoor Positioning and Indoor Navigation (IPIN) conference holds an annual competition in which indoor localization systems from different research groups worldwide are evaluated empirically. The objective of this competition is to establish a systematic evaluation methodology with rigorous metrics both for real-Time (on-site) and post-processing (off-site) situations, in a realistic environment unfamiliar to the prototype developers. For the IPIN 2018 conference, this competition was held on September 22nd, 2018, in Atlantis, a large shopping mall in Nantes (France). Four competition tracks (two on-site and two off-site) were designed. They consisted of several 1 km routes traversing several floors of the mall. Along these paths, 180 points were topographically surveyed with a 10 cm accuracy, to serve as ground truth landmarks, combining theodolite measurements, differential global navigation satellite system (GNSS) and 3D scanner systems. 34 teams effectively competed. The accuracy score corresponds to the third quartile (75th percentile) of an error metric that combines the horizontal positioning error and the floor detection. The best results for the on-site tracks showed an accuracy score of 11.70 m (Track 1) and 5.50 m (Track 2), while the best results for the off-site tracks showed an accuracy score of 0.90 m (Track 3) and 1.30 m (Track 4). These results showed that it is possible to obtain high accuracy indoor positioning solutions in large, realistic environments using wearable light-weight sensors without deploying any beacon. This paper describes the organization work of the tracks, analyzes the methodology used to quantify the results, reviews the lessons learned from the competition and discusses its future.",

keywords = "IPIN, competition, indoor positioning, smartphone",

author = "Valerie Renaudin and Miguel Ortiz and Johan Perul and Joaquin Torres-Sospedra and Jimenez, {Antonio Ramon} and Antoni Perez-Navarro and {Martin Mendoza-Silva}, German and Fernando Seco and Yael Landau and Revital Marbel and Boaz Ben-Moshe and Xingyu Zheng and Feng Ye and Jian Kuang and Yu Li and Xiaoji Niu and Vlad Landa and Shlomi Hacohen and Nir Shvalb and Chuanhua Lu and Hideaki Uchiyama and Diego Thomas and Atsushi Shimada and Taniguchi, {Rin Ichiro} and Zhenxing Ding and Feng Xu and Nikolai Kronenwett and Blagovest Vladimirov and Soyeon Lee and Eunyoung Cho and Sungwoo Jun and Changeun Lee and Sangjoon Park and Yonghyun Lee and Jehyeok Rew and Changjun Park and Hyeongyo Jeong and Jaeseung Han and Keumryeol Lee and Wenchao Zhang and Xianghong Li and Dongyan Wei and Ying Zhang and Park, {So Young} and Park, {Chan Gook} and Stefan Knauth and Georgios Pipelidis and Nikolaos Tsiamitros and Tomas Lungenstrass and Morales, {Juan Pablo} and Jens Trogh and David Plets and Miroslav Opiela and Fang, {Shih Hau} and Yu Tsao and Chien, {Ying Ren} and Yang, {Shi Shen} and Ye, {Shih Jyun} and Ali, {Muhammad Usman} and Soojung Hur and Yongwan Park",

note = "Publisher Copyright: {\textcopyright} 2013 IEEE.",

year = "2019",

doi = "10.1109/ACCESS.2019.2944389",

language = "אנגלית",

volume = "7",

pages = "148594--148628",

journal = "IEEE Access",

issn = "2169-3536",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

Renaudin, V, Ortiz, M, Perul, J, Torres-Sospedra, J, Jimenez, AR, Perez-Navarro, A, Martin Mendoza-Silva, G, Seco, F, Landau, Y, Marbel, R, Ben-Moshe, B, Zheng, X, Ye, F, Kuang, J, Li, Y, Niu, X, Landa, V, Hacohen, S, Shvalb, N, Lu, C, Uchiyama, H, Thomas, D, Shimada, A, Taniguchi, RI, Ding, Z, Xu, F, Kronenwett, N, Vladimirov, B, Lee, S, Cho, E, Jun, S, Lee, C, Park, S, Lee, Y, Rew, J, Park, C, Jeong, H, Han, J, Lee, K, Zhang, W, Li, X, Wei, D, Zhang, Y, Park, SY, Park, CG, Knauth, S, Pipelidis, G, Tsiamitros, N, Lungenstrass, T, Morales, JP, Trogh, J, Plets, D, Opiela, M, Fang, SH, Tsao, Y, Chien, YR, Yang, SS, Ye, SJ, Ali, MU, Hur, S & Park, Y 2019, 'Evaluating Indoor Positioning Systems in a Shopping Mall: The Lessons Learned from the IPIN 2018 Competition', IEEE Access, vol. 7, 8852722, pp. 148594-148628. https://doi.org/10.1109/ACCESS.2019.2944389

TY - JOUR

T1 - Evaluating Indoor Positioning Systems in a Shopping Mall

T2 - The Lessons Learned from the IPIN 2018 Competition

AU - Renaudin, Valerie

AU - Ortiz, Miguel

AU - Perul, Johan

AU - Torres-Sospedra, Joaquin

AU - Jimenez, Antonio Ramon

AU - Perez-Navarro, Antoni

AU - Martin Mendoza-Silva, German

AU - Seco, Fernando

AU - Landau, Yael

AU - Marbel, Revital

AU - Ben-Moshe, Boaz

AU - Zheng, Xingyu

AU - Ye, Feng

AU - Kuang, Jian

AU - Li, Yu

AU - Niu, Xiaoji

AU - Landa, Vlad

AU - Hacohen, Shlomi

AU - Shvalb, Nir

AU - Lu, Chuanhua

AU - Uchiyama, Hideaki

AU - Thomas, Diego

AU - Shimada, Atsushi

AU - Taniguchi, Rin Ichiro

AU - Ding, Zhenxing

AU - Xu, Feng

AU - Kronenwett, Nikolai

AU - Vladimirov, Blagovest

AU - Lee, Soyeon

AU - Cho, Eunyoung

AU - Jun, Sungwoo

AU - Lee, Changeun

AU - Park, Sangjoon

AU - Lee, Yonghyun

AU - Rew, Jehyeok

AU - Park, Changjun

AU - Jeong, Hyeongyo

AU - Han, Jaeseung

AU - Lee, Keumryeol

AU - Zhang, Wenchao

AU - Li, Xianghong

AU - Wei, Dongyan

AU - Zhang, Ying

AU - Park, So Young

AU - Park, Chan Gook

AU - Knauth, Stefan

AU - Pipelidis, Georgios

AU - Tsiamitros, Nikolaos

AU - Lungenstrass, Tomas

AU - Morales, Juan Pablo

AU - Trogh, Jens

AU - Plets, David

AU - Opiela, Miroslav

AU - Fang, Shih Hau

AU - Tsao, Yu

AU - Chien, Ying Ren

AU - Yang, Shi Shen

AU - Ye, Shih Jyun

AU - Ali, Muhammad Usman

AU - Hur, Soojung

AU - Park, Yongwan

PY - 2019

Y1 - 2019

N2 - The Indoor Positioning and Indoor Navigation (IPIN) conference holds an annual competition in which indoor localization systems from different research groups worldwide are evaluated empirically. The objective of this competition is to establish a systematic evaluation methodology with rigorous metrics both for real-Time (on-site) and post-processing (off-site) situations, in a realistic environment unfamiliar to the prototype developers. For the IPIN 2018 conference, this competition was held on September 22nd, 2018, in Atlantis, a large shopping mall in Nantes (France). Four competition tracks (two on-site and two off-site) were designed. They consisted of several 1 km routes traversing several floors of the mall. Along these paths, 180 points were topographically surveyed with a 10 cm accuracy, to serve as ground truth landmarks, combining theodolite measurements, differential global navigation satellite system (GNSS) and 3D scanner systems. 34 teams effectively competed. The accuracy score corresponds to the third quartile (75th percentile) of an error metric that combines the horizontal positioning error and the floor detection. The best results for the on-site tracks showed an accuracy score of 11.70 m (Track 1) and 5.50 m (Track 2), while the best results for the off-site tracks showed an accuracy score of 0.90 m (Track 3) and 1.30 m (Track 4). These results showed that it is possible to obtain high accuracy indoor positioning solutions in large, realistic environments using wearable light-weight sensors without deploying any beacon. This paper describes the organization work of the tracks, analyzes the methodology used to quantify the results, reviews the lessons learned from the competition and discusses its future.

AB - The Indoor Positioning and Indoor Navigation (IPIN) conference holds an annual competition in which indoor localization systems from different research groups worldwide are evaluated empirically. The objective of this competition is to establish a systematic evaluation methodology with rigorous metrics both for real-Time (on-site) and post-processing (off-site) situations, in a realistic environment unfamiliar to the prototype developers. For the IPIN 2018 conference, this competition was held on September 22nd, 2018, in Atlantis, a large shopping mall in Nantes (France). Four competition tracks (two on-site and two off-site) were designed. They consisted of several 1 km routes traversing several floors of the mall. Along these paths, 180 points were topographically surveyed with a 10 cm accuracy, to serve as ground truth landmarks, combining theodolite measurements, differential global navigation satellite system (GNSS) and 3D scanner systems. 34 teams effectively competed. The accuracy score corresponds to the third quartile (75th percentile) of an error metric that combines the horizontal positioning error and the floor detection. The best results for the on-site tracks showed an accuracy score of 11.70 m (Track 1) and 5.50 m (Track 2), while the best results for the off-site tracks showed an accuracy score of 0.90 m (Track 3) and 1.30 m (Track 4). These results showed that it is possible to obtain high accuracy indoor positioning solutions in large, realistic environments using wearable light-weight sensors without deploying any beacon. This paper describes the organization work of the tracks, analyzes the methodology used to quantify the results, reviews the lessons learned from the competition and discusses its future.

KW - IPIN

KW - competition

KW - indoor positioning

KW - smartphone

UR - http://www.scopus.com/inward/record.url?scp=85077705668&partnerID=8YFLogxK

U2 - 10.1109/ACCESS.2019.2944389

DO - 10.1109/ACCESS.2019.2944389

M3 - ???researchoutput.researchoutputtypes.contributiontojournal.article???

AN - SCOPUS:85077705668

SN - 2169-3536

VL - 7

SP - 148594

EP - 148628

JO - IEEE Access

JF - IEEE Access

M1 - 8852722

ER -

Evaluating Indoor Positioning Systems in a Shopping Mall: The Lessons Learned from the IPIN 2018 Competition

Abstract

Keywords

Access to Document

Other files and links

Fingerprint

Cite this