Header logo is pi
36 results

2017


Thumb xl publications toc
Magnetically actuated soft capsule endoscope for fine-needle aspiration biopsy

Son, D., Dogan, M. D., Sitti, M.

In Proceedings 2017 IEEE International Conference on Robotics and Automation (ICRA), pages: 1132-1139, IEEE, Piscataway, NJ, USA, IEEE International Conference on Robotics and Automation (ICRA), May 2017 (inproceedings)

Abstract
This paper presents a magnetically actuated soft capsule endoscope for fine-needle aspiration biopsy (B-MASCE) in the upper gastrointestinal tract. A thin and hollow needle is attached to the capsule, which can penetrate deeply into tissues to obtain subsurface biopsy sample. The design utilizes a soft elastomer body as a compliant mechanism to guide the needle. An internal permanent magnet provides a means for both actuation and tracking. The capsule is designed to roll towards its target and then deploy the biopsy needle in a precise location selected as the target area. B-MASCE is controlled by multiple custom-designed electromagnets while its position and orientation are tracked by a magnetic sensor array. In in vitro trials, B-MASCE demonstrated rolling locomotion and biopsy of a swine tissue model positioned inside an anatomical human stomach model. It was confirmed after the experiment that a tissue sample was retained inside the needle.

DOI Project Page [BibTex]

2017

DOI Project Page [BibTex]


Thumb xl publications toc
Endo-VMFuseNet: Deep Visual-Magnetic Sensor Fusion Approach for Uncalibrated, Unsynchronized and Asymmetric Endoscopic Capsule Robot Localization Data

Turan, M., Almalioglu, Y., Gilbert, H., Eren Sari, A., Soylu, U., Sitti, M.

ArXiv e-prints, September 2017 (article)

Abstract
In the last decade, researchers and medical device companies have made major advances towards transforming passive capsule endoscopes into active medical robots. One of the major challenges is to endow capsule robots with accurate perception of the environment inside the human body, which will provide necessary information and enable improved medical procedures. We extend the success of deep learning approaches from various research fields to the problem of uncalibrated, asynchronous, and asymmetric sensor fusion for endoscopic capsule robots. The results performed on real pig stomach datasets show that our method achieves sub-millimeter precision for both translational and rotational movements and contains various advantages over traditional sensor fusion techniques.

link (url) Project Page [BibTex]


Thumb xl publications toc
EndoSensorFusion: Particle Filtering-Based Multi-sensory Data Fusion with Switching State-Space Model for Endoscopic Capsule Robots

Turan, M., Almalioglu, Y., Gilbert, H., Araujo, H., Cemgil, T., Sitti, M.

ArXiv e-prints, September 2017 (article)

Abstract
A reliable, real time multi-sensor fusion functionality is crucial for localization of actively controlled capsule endoscopy robots, which are an emerging, minimally invasive diagnostic and therapeutic technology for the gastrointestinal (GI) tract. In this study, we propose a novel multi-sensor fusion approach based on a particle filter that incorporates an online estimation of sensor reliability and a non-linear kinematic model learned by a recurrent neural network. Our method sequentially estimates the true robot pose from noisy pose observations delivered by multiple sensors. We experimentally test the method using 5 degree-of-freedom (5-DoF) absolute pose measurement by a magnetic localization system and a 6-DoF relative pose measurement by visual odometry. In addition, the proposed method is capable of detecting and handling sensor failures by ignoring corrupted data, providing the robustness expected of a medical device. Detailed analyses and evaluations are presented using ex-vivo experiments on a porcine stomach model prove that our system achieves high translational and rotational accuracies for different types of endoscopic capsule robot trajectories.

link (url) Project Page [BibTex]


Thumb xl publications toc
Sparse-then-Dense Alignment based 3D Map Reconstruction Method for Endoscopic Capsule Robots

Turan, M., Yigit Pilavci, Y., Ganiyusufoglu, I., Araujo, H., Konukoglu, E., Sitti, M.

ArXiv e-prints, August 2017 (article)

Abstract
Since the development of capsule endoscopcy technology, substantial progress were made in converting passive capsule endoscopes to robotic active capsule endoscopes which can be controlled by the doctor. However, robotic capsule endoscopy still has some challenges. In particular, the use of such devices to generate a precise and globally consistent three-dimensional (3D) map of the entire inner organ remains an unsolved problem. Such global 3D maps of inner organs would help doctors to detect the location and size of diseased areas more accurately, precisely, and intuitively, thus permitting more accurate and intuitive diagnoses. The proposed 3D reconstruction system is built in a modular fashion including preprocessing, frame stitching, and shading-based 3D reconstruction modules. We propose an efficient scheme to automatically select the key frames out of the huge quantity of raw endoscopic images. Together with a bundle fusion approach that aligns all the selected key frames jointly in a globally consistent way, a significant improvement of the mosaic and 3D map accuracy was reached. To the best of our knowledge, this framework is the first complete pipeline for an endoscopic capsule robot based 3D map reconstruction containing all of the necessary steps for a reliable and accurate endoscopic 3D map. For the qualitative evaluations, a real pig stomach is employed. Moreover, for the first time in literature, a detailed and comprehensive quantitative analysis of each proposed pipeline modules is performed using a non-rigid esophagus gastro duodenoscopy simulator, four different endoscopic cameras, a magnetically activated soft capsule robot (MASCE), a sub-millimeter precise optical motion tracker and a fine-scale 3D optical scanner.

link (url) Project Page [BibTex]

link (url) Project Page [BibTex]


Thumb xl publications toc
Deep EndoVO: A Recurrent Convolutional Neural Network (RCNN) based Visual Odometry Approach for Endoscopic Capsule Robots

Turan, M., Almalioglu, Y., Araujo, H., Konukoglu, E., Sitti, M.

ArXiv e-prints, 2017 (article)

Abstract
Ingestible wireless capsule endoscopy is an emerging minimally invasive diagnostic technology for inspection of the GI tract and diagnosis of a wide range of diseases and pathologies. Medical device companies and many research groups have recently made substantial progresses in converting passive capsule endoscopes to active capsule robots, enabling more accurate, precise, and intuitive detection of the location and size of the diseased areas. Since a reliable real time pose estimation functionality is crucial for actively controlled endoscopic capsule robots, in this study, we propose a monocular visual odometry (VO) method for endoscopic capsule robot operations. Our method lies on the application of the deep Recurrent Convolutional Neural Networks (RCNNs) for the visual odometry task, where Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) are used for the feature extraction and inference of dynamics across the frames, respectively. Detailed analyses and evaluations made on a real pig stomach dataset proves that our system achieves high translational and rotational accuracies for different types of endoscopic capsule robot trajectories.

link (url) Project Page [BibTex]


Thumb xl imagetoc
A Deep Learning Based 6 Degree-of-Freedom Localization Method for Endoscopic Capsule Robots

Turan, M., Almalioglu, Y., Konukoglu, E., Sitti, M.

arXiv preprint arXiv:1705.05435, 2017 (article)

Abstract
We present a robust deep learning based 6 degrees-of-freedom (DoF) localization system for endoscopic capsule robots. Our system mainly focuses on localization of endoscopic capsule robots inside the GI tract using only visual information captured by a mono camera integrated to the robot. The proposed system is a 23-layer deep convolutional neural network (CNN) that is capable to estimate the pose of the robot in real time using a standard CPU. The dataset for the evaluation of the system was recorded inside a surgical human stomach model with realistic surface texture, softness, and surface liquid properties so that the pre-trained CNN architecture can be transferred confidently into a real endoscopic scenario. An average error of 7.1% and 3.4% for translation and rotation has been obtained, respectively. The results accomplished from the experiments demonstrate that a CNN pre-trained with raw 2D endoscopic images performs accurately inside the GI tract and is robust to various challenges posed by reflection distortions, lens imperfections, vignetting, noise, motion blur, low resolution, and lack of unique landmarks to track.

link (url) Project Page [BibTex]


Thumb xl image toc
A Non-Rigid Map Fusion-Based RGB-Depth SLAM Method for Endoscopic Capsule Robots

Turan, M., Almalioglu, Y., Araujo, H., Konukoglu, E., Sitti, M.

arXiv preprint arXiv:1705.05444, May 2017 (article)

Abstract
In the gastrointestinal (GI) tract endoscopy field, ingestible wireless capsule endoscopy is considered as a minimally invasive novel diagnostic technology to inspect the entire GI tract and to diagnose various diseases and pathologies. Since the development of this technology, medical device companies and many groups have made significant progress to turn such passive capsule endoscopes into robotic active capsule endoscopes to achieve almost all functions of current active flexible endoscopes. However, the use of robotic capsule endoscopy still has some challenges. One such challenge is the precise localization of such active devices in 3D world, which is essential for a precise three-dimensional (3D) mapping of the inner organ. A reliable 3D map of the explored inner organ could assist the doctors to make more intuitive and correct diagnosis. In this paper, we propose to our knowledge for the first time in literature a visual simultaneous localization and mapping (SLAM) method specifically developed for endoscopic capsule robots. The proposed RGB-Depth SLAM method is capable of capturing comprehensive dense globally consistent surfel-based maps of the inner organs explored by an endoscopic capsule robot in real time. This is achieved by using dense frame-to-model camera tracking and windowed surfelbased fusion coupled with frequent model refinement through non-rigid surface deformations.

link (url) Project Page [BibTex]

link (url) Project Page [BibTex]


Thumb xl a fully toc
A fully dense and globally consistent 3D map reconstruction approach for GI tract to enhance therapeutic relevance of the endoscopic capsule robot

Turan, M., Pilavci, Y. Y., Jamiruddin, R., Araujo, H., Konukoglu, E., Sitti, M.

arXiv preprint arXiv:1705.06524, 2017 (article)

Abstract
In the gastrointestinal (GI) tract endoscopy field, ingestible wireless capsule endoscopy is emerging as a novel, minimally invasive diagnostic technology for inspection of the GI tract and diagnosis of a wide range of diseases and pathologies. Since the development of this technology, medical device companies and many research groups have made substantial progress in converting passive capsule endoscopes to robotic active capsule endoscopes with most of the functionality of current active flexible endoscopes. However, robotic capsule endoscopy still has some challenges. In particular, the use of such devices to generate a precise three-dimensional (3D) mapping of the entire inner organ remains an unsolved problem. Such global 3D maps of inner organs would help doctors to detect the location and size of diseased areas more accurately and intuitively, thus permitting more reliable diagnoses. To our knowledge, this paper presents the first complete pipeline for a complete 3D visual map reconstruction of the stomach. The proposed pipeline is modular and includes a preprocessing module, an image registration module, and a final shape-from-shading-based 3D reconstruction module; the 3D map is primarily generated by a combination of image stitching and shape-from-shading techniques, and is updated in a frame-by-frame iterative fashion via capsule motion inside the stomach. A comprehensive quantitative analysis of the proposed 3D reconstruction method is performed using an esophagus gastro duodenoscopy simulator, three different endoscopic cameras, and a 3D optical scanner.

link (url) Project Page [BibTex]


Thumb xl image toc
Six Degree-of-Freedom Localization of Endoscopic Capsule Robots using Recurrent Neural Networks embedded into a Convolutional Neural Network

Turan, M., Abdullah, A., Jamiruddin, R., Araujo, H., Konukoglu, E., Sitti, M.

arXiv preprint arXiv:1705.06196, May 2017 (article)

Abstract
Since its development, ingestible wireless endoscopy is considered to be a painless diagnostic method to detect a number of diseases inside GI tract. Medical related engineering companies have made significant improvements in this technology in last decade; however, some major limitations still residue. Localization of the next generation steerable endoscopic capsule robot in six degreeof-freedom (DoF) and active motion control are some of these limitations. The significance of localization capability concerns with the doctors correct diagnosis of the disease area. This paper presents a very robust 6-DoF localization method based on supervised training of an architecture consisting of recurrent networks (RNN) embedded into a convolutional neural network (CNN) to make use of both just-in-moment information obtained by CNN and correlative information across frames obtained by RNN. To our knowledge, our idea of embedding RNNs into a CNN architecture is for the first time proposed in literature. The experimental results show that the proposed RNN-in-CNN architecture performs very well for endoscopic capsule robot localization in cases vignetting, reflection distortions, noise, sudden camera movements and lack of distinguishable features.

DOI Project Page [BibTex]

2016


Thumb xl pnas 2016 lum 1608193113 2
Shape-programmable magnetic soft matter

Lum, G. Z., Ye, Z., Dong, X., Marvi, H., Erin, O., Hu, W., Sitti, M.

Proceedings of the National Academy of Sciences, pages: 201608193, National Acad Sciences, May 2016 (article)

Abstract
Shape-programmable matter is a class of active materials whose geometry can be controlled to potentially achieve mechanical functionalities beyond those of traditional machines. Among these materials, magnetically actuated matter is particularly promising for achieving complex time-varying shapes at small scale (overall dimensions smaller than 1 cm). However, previous work can only program these materials for limited applications, as they rely solely on human intuition to approximate the required magnetization profile and actuating magnetic fields for their materials. Here, we propose a universal programming methodology that can automatically generate the required magnetization profile and actuating fields for soft matter to achieve new time-varying shapes. The universality of the proposed method can therefore inspire a vast number of miniature soft devices that are critical in robotics, smart engineering surfaces and materials, and biomedical devices. Our proposed method includes theoretical formulations, computational strategies, and fabrication procedures for programming magnetic soft matter. The presented theory and computational method are universal for programming 2D or 3D time-varying shapes, whereas the fabrication technique is generic only for creating planar beams. Based on the proposed programming method, we created a jellyfish-like robot, a spermatozoid-like undulating swimmer, and an artificial cilium that could mimic the complex beating patterns of its biological counterpart.

DOI Project Page Project Page [BibTex]

2016

DOI Project Page Project Page [BibTex]

2015


Thumb xl 07293677
A 5-D localization method for a magnetically manipulated untethered robot using a 2-D array of Hall-effect sensors

Son, D., Yim, S., Sitti, M.

IEEE/ASME Transactions on Mechatronics, 21(2):708-716, IEEE, October 2015 (article)

Abstract
This paper introduces a new five-dimensional localization method for an untethered meso-scale magnetic robot, which is manipulated by a computer-controlled electromagnetic system. The developed magnetic localization setup is a two-dimensional array of mono-axial Hall-effect sensors, which measure the perpendicular magnetic fields at their given positions. We introduce two steps for localizing a magnetic robot more accurately. First, the dipole modeled magnetic field of the electromagnet is subtracted from the measured data in order to determine the robot's magnetic field. Secondly, the subtracted magnetic field is twice differentiated in the perpendicular direction of the array, so that the effect of the electromagnetic field in the localization process is minimized. Five variables regarding the position and orientation of the robot are determined by minimizing the error between the measured magnetic field and the modeled magnetic field in an optimization method. The resulting position error is 2.1±0.8 mm and angular error is 6.7±4.3° within the applicable range (5 cm) of magnetic field sensors at 200 Hz. The proposed localization method would be used for the position feedback control of untethered magnetic devices or robots for medical applications in the future.

DOI Project Page [BibTex]

2015


Thumb xl publications toc
Biomedical applications of untethered mobile milli/microrobots

Sitti, M., Ceylan, H., Hu, W., Giltinan, J., Turan, M., Yim, S., Diller, E.

Proceedings of the IEEE, 103(2):205-224, IEEE, March 2015 (article)

Abstract
Untethered robots miniaturized to the length scale of millimeter and below attract growing attention for the prospect of transforming many aspects of health care and bioengineering. As the robot size goes down to the order of a single cell, previously inaccessible body sites would become available for high-resolution in situ and in vivo manipulations. This unprecedented direct access would enable an extensive range of minimally invasive medical operations. Here, we provide a comprehensive review of the current advances in biomedical untethered mobile milli/microrobots. We put a special emphasis on the potential impacts of biomedical microrobots in the near future. Finally, we discuss the existing challenges and emerging concepts associated with designing such a miniaturized robot for operation inside a biological environment for biomedical applications.

DOI Project Page Project Page [BibTex]

DOI Project Page Project Page [BibTex]

2014


no image
MultiMo-Bat: A biologically inspired integrated jumping–gliding robot

Woodward, M. A., Sitti, M.

The International Journal of Robotics Research, 33(12):1511-1529, SAGE Publications Sage UK: London, England, 2014 (article)

Project Page [BibTex]

2014

Project Page [BibTex]


no image
Biopsy using a Magnetic Capsule Endoscope Carrying, Releasing and Retrieving Untethered Micro-Grippers

Yim, S., Gultepe, E., Gracias, D. H., Sitti, M.

IEEE Trans. on Biomedical Engineering, 61(2):513-521, IEEE, 2014 (article)

Project Page [BibTex]

Project Page [BibTex]


no image
SoftCubes: Stretchable and self-assembling three-dimensional soft modular matter

Yim, S., Sitti, M.

The International Journal of Robotics Research, 33(8):1083-1097, SAGE Publications Sage UK: London, England, 2014 (article)

Project Page Project Page [BibTex]

Project Page Project Page [BibTex]


no image
Liftoff of a Motor-Driven, Flapping-Wing Microaerial Vehicle Capable of Resonance

Hines, L., Campolo, D., Sitti, M.

IEEE Trans. on Robotics, 30(1):220-232, IEEE, 2014 (article)

Project Page [BibTex]

Project Page [BibTex]

2013


no image
Magnetically Actuated Soft Capsule With the Multimodal Drug Release Function

Yim, S., Goyal, K., Sitti, M.

IEEE/ASME Trans. on Mechatronics, 18(4):1413-1418, IEEE, 2013 (article)

Project Page [BibTex]

2013

Project Page [BibTex]


no image
SoftCubes: towards a soft modular matter

Yim, S., Sitti, M.

In Robotics and Automation (ICRA), 2013 IEEE International Conference on, pages: 530-536, 2013 (inproceedings)

Project Page [BibTex]

Project Page [BibTex]


no image
A simulation and design tool for a passive rotation flapping wing mechanism

Arabagi, V., Hines, L., Sitti, M.

IEEE/ASME Transactions on Mechatronics, 18(2):787-798, 2013 (article)

Project Page [BibTex]

Project Page [BibTex]

2012


no image
Shape-Programmable Soft Capsule Robots for Semi-Implantable Drug Delivery

Yim, S., Sitti, M.

Mechatronics, IEEE/ASME Transactions on, 2012 (article)

Project Page [BibTex]

2012

Project Page [BibTex]


no image
Design and rolling locomotion of a magnetically actuated soft capsule endoscope

Yim, S., Sitti, M.

IEEE Transactions on Robotics, 28(1):183-194, IEEE, 2012 (article)

Project Page [BibTex]

Project Page [BibTex]

2011


no image
Design of a miniature integrated multi-modal jumping and gliding robot

Woodward, M. A., Sitti, M.

In Intelligent Robots and Systems (IROS), 2011 IEEE/RSJ International Conference on, pages: 556-561, 2011 (inproceedings)

Project Page [BibTex]

2011

Project Page [BibTex]


no image
Biaxial mechanical modeling of the small intestine

Bellini, C., Glass, P., Sitti, M., Di Martino, E. S.

Journal of the mechanical behavior of biomedical materials, 4(8):1727-1740, Elsevier, 2011 (article)

Project Page [BibTex]

Project Page [BibTex]


no image
Free flight simulations and pitch and roll control experiments of a sub-gram flapping-flight micro aerial vehicle

Hines, L. L., Arabagi, V., Sitti, M.

In Robotics and Automation (ICRA), 2011 IEEE International Conference on, pages: 1-7, 2011 (inproceedings)

Project Page [BibTex]

Project Page [BibTex]

2010


no image
Roll and pitch motion analysis of a biologically inspired quadruped water runner robot

Park, H. S., Floyd, S., Sitti, M.

The International Journal of Robotics Research, 29(10):1281-1297, SAGE Publications Sage UK: London, England, 2010 (article)

Project Page [BibTex]

2010

Project Page [BibTex]


no image
Control performance simulation in the design of a flapping wing micro-aerial vehicle

Hines, L. L., Arabagi, V., Sitti, M.

In Intelligent Robots and Systems (IROS), 2010 IEEE/RSJ International Conference on, pages: 1090-1095, 2010 (inproceedings)

Project Page [BibTex]

Project Page [BibTex]

2009


no image
Compliant footpad design analysis for a bio-inspired quadruped amphibious robot

Park, H. S., Sitti, M.

In Intelligent Robots and Systems, 2009. IROS 2009. IEEE/RSJ International Conference on, pages: 645-651, 2009 (inproceedings)

Project Page [BibTex]

2009

Project Page [BibTex]


no image
Dynamic modeling and analysis of pitch motion of a basilisk lizard inspired quadruped robot running on water

Park, H. S., Floyd, S., Sitti, M.

In Robotics and Automation, 2009. ICRA’09. IEEE International Conference on, pages: 2655-2660, 2009 (inproceedings)

Project Page [BibTex]

Project Page [BibTex]


no image
A Swallowable Tethered Capsule Endoscope for Diagnosing Barrett’s Esophagus

Glass, P., Sitti, M., Pennathur, A., Appasamy, R.

Gastrointestinal Endoscopy, 69(5):AB106, Mosby, 2009 (article)

Project Page [BibTex]

Project Page [BibTex]

2008


no image
Simulation and analysis of a passive pitch reversal flapping wing mechanism for an aerial robotic platform

Arabagi, V., Sitti, M.

In Intelligent Robots and Systems, 2008. IROS 2008. IEEE/RSJ International Conference on, pages: 1260-1265, 2008 (inproceedings)

Project Page [BibTex]

2008

Project Page [BibTex]


no image
A legged anchoring mechanism for capsule endoscopes using micropatterned adhesives

Glass, P., Cheung, E., Sitti, M.

IEEE Transactions on Biomedical Engineering, 55(12):2759-2767, IEEE, 2008 (article)

Project Page [BibTex]

Project Page [BibTex]


no image
Performance of different foot designs for a water running robot

Floyd, S., Adilak, S., Ramirez, S., Rogman, R., Sitti, M.

In Robotics and Automation, 2008. ICRA 2008. IEEE International Conference on, pages: 244-250, 2008 (inproceedings)

Project Page [BibTex]

Project Page [BibTex]

2006


no image
A novel water running robot inspired by basilisk lizards

Floyd, S., Keegan, T., Palmisano, J., Sitti, M.

In Intelligent Robots and Systems, 2006 IEEE/RSJ International Conference on, pages: 5430-5436, 2006 (inproceedings)

Project Page [BibTex]

2006

Project Page [BibTex]

2005


no image
A new endoscopic microcapsule robot using beetle inspired microfibrillar adhesives

Cheung, E., Karagozler, M. E., Park, S., Kim, B., Sitti, M.

In Advanced Intelligent Mechatronics. Proceedings, 2005 IEEE/ASME International Conference on, pages: 551-557, 2005 (inproceedings)

Project Page [BibTex]

2005

Project Page [BibTex]