DEXTRA SMART: Driverless Mobile App. Run Car
Amiraj Charitable Trust Established, Amiraj College of Engineering and Technology in the year 2012. The College is approved by AICTE and GTU as well as ISO 9001:2008 certified. With currently 600 students studying Engineering in five different branches namely Civil, Mechanical, Computer, Electrical and EC. We are a forward looking, modern engineering institution with an objective to provide high quality education and focus on applied research. The institute currently is focusing on Science beyond Engineering by providing exclusive Skill development training programs for its existing students. The extra exposure with respect to skill development (Over and above Engineering) is not only making them conscious about skills and personality but it’s also helping the youth to align themselves with Honorable PRIME MINISTER OF INDIA – “Shri Narendrabhai Modi’s” vision of “MAKE IN INDIA” CAMPAIGN.
Science Beyond Engineering ismost innovatively designed skill set training program to provide completely life changing superior skill sets, to the ambitious Amiraj engineering generation of Gujarat. The Skill development training campaign integrates Engineering knowledge with world class Communication, Management and Marketing skills, keeping in mind the industry demands, corporate ethics, global competition and changing trends. The education, experience and environment in the training programprovides students an edge by transforming them into personalities showcasing positive attitude, intellectually stimulating knowledge, world class communication skills, Meticulous Managerial skills and Creative Marketing Mantras.Picturing a brief idea about the punch line that we bear, Science beyond Engineering one can go by the literal meaning of these words. Engineers and scientists invent, Entrepreneurs commercialize it, and finally we, laymen, use it. While in all these hustle and bustle, Amiraj Group thought of a little out of the box, of not keeping the science confined to the realms of the engineering only, rather extending our knowledge to the entire scope of science and taking it beyond engineering.Science beyond engineering is creating a platform for Amiraj faculties and students to exhibit and focus on skill sets that matter. Skill sets like leadership, innovations, communication, listening, management, teamwork, stress management and various other personality skill sets that will necessarily help them gain superiority and edge in all their professional endeavors. The institution under its drive for innovation believes in the idea of ‘You see it; we shape it.’ So as a part of innovation drive where the youth is pushed to not only learn engineering but also to create their own engineering.These Amiraj fraternity, spearheaded by faculty supported by students came together with the idea of AmirajDextra Smart: Driverless mobile app. run car. This little idea has triggered its shots at a larger platform. The out of the box idea, of innovations, of enhancement is at the roots of the various ongoing projects at Amiraj and the one major innovation that we are going to talk aboutAmirajDextra Smart: Driverless mobile app.run car.
Autonomous vehicles are becoming the talk of the town in the automation world. Autonomous vehicles are the ones which can drive themselves without human supervision or input. While the unmanned vehicles are the vehicles which are either controlled remotely or perhaps operate autonomously. Vehicles can also operate semi autonomously; taking some control of the aspects of their driving, whilst the human driver can retain control over the other factors. This smart car is an Android application based controlled car, and can be classified as autonomous unmanned car controlled and operated remotely. The ‘Driver Only’ is the autonomy level for the car where the vehicle is entirely under human control but may have some automated systems.The design of the technology of the car is such that its usage can be extended to the sophisticated artificial intelligence and robotics sector, alongside for doing jobs which are dangerous for humans. All in all it has a very long list of the fields of application. The basic idea of the project was to design the driverless car for the Indian road system. It could have been much simpler had we used the GPS and Video surveillance system, but the obstacle here is, to use GPS one needs a properly defined mapping system of the area which is not up to the mark yet.
Autonomous cars are robots capable of unmanned driving in an unknown urban area. Research in autonomous cars have received a broader interest in recent years and unfolded many insights for robot systems in different applications. The industry, especially automobile manufacturers, are eager to improve advanced driver assistance systems (ADAS), such as lane departure warning and intelligent speed adaptation systems while the military is strongly interested in unmanned drones for military purposes. From a Computer Science perspective autonomous vehicles function as a research foundation for progress in machine learning, computer vision, fusion of sensor data, path planning, decision making and intelligent autonomous behavior.
While unmanned driving is the ultimate goal, in the development process of autonomous vehicles human drivers must be present in case of failures. A remote control is a perfect instrument to send commands to the vehicles as well as receiving status data from without actually remaining in the car. Smart Mobile phones possess the necessary computing capabilities and sensor functionalities to act as a remote control. The remote can toggle between completely autonomous mode, where the phone acts as a mobile monitoring panel, and manual mode, where the operator can use the phone to steer and drive the car. A human operator sent commands for the desired position of steering wheel, brake pedal as well as gear shift signals to the car. Additionally, a video stream from mounted cameras on top of the car is broadcasted back to the Smart mobile Phone as feedback.
A remote control is typical for client-server architecture whereas the client software resides in the remote sending out control signals to the server system to be controlled. The server is responsible to accept or reject these commands and execute them accordingly. As of a feedback service the server may return an answer back to the client.
Several sensors are used and mounted on top of the car: GPS antenna give information about the position and direction of the car. Two video cameras are mounted in front of the car for stereo-vision modules to detect lane markings and roadside. One of the camera broadcasts its video stream to the smart phone to provide a view of where the car is heading. Sensors are used to sweep the surroundings for any obstacles that need to be evaded. All sensor data are collected and fashioned into one state model about the vehicle itself and its surroundings that is representing the perception of the car.
A server provides the necessary computing power for the running software modules, the actual intelligence of the vehicle. Here, the fashioned sensor data are used to make decisions on what action needs to be executed next given the current situation. The necessary commands are then transmitted to actuators in the steering wheel and brake pedals to execute the made decision. When the smart phone acts as a remote control in manual mode, the intelligence in the car is turned off, and the commands given from the phone are directly transmitted to the actuators.
The web application software was developed for smart phone. The multi-touch screen of the smart phone can be used to capture multiple commands with more than one finger simultaneously, e.g. pushing break and steering to the left. The user interface of web application software is similar to many available race games for the smart phone: Two touch sensitive buttons represent accelerator and brake control, touching them will send out commands either to give accelerator or to brake. If both buttons are pressed, the brake control overrides the accelerator control. The steering wheel is controlled by two buttons. Since the car is an automatic, the gear can only be switched between reverse, park and drive. Holding the brake is required for a gear shift. The Smart phone possesses a 1.2GHz dual-core processor graphics processor, 1GB of RAM, communication HSPA, EDGE/GPRS and Wi-Fi 802.11 Dual-band support (2.4GHz, 5GHz) with 802.11n Wi-Fi Channel Bonding .This graphics processor is used to display the camera image from the video camera of the car onto the main screen.
Communication from smart Phone to car:
The communication between smart phone and car is based on simple Wi-Fi and 3G technology. The transmission from the phone to the car requires the values for the desired accelerator or brake position, steering wheel position and desired gear activity. All this information is packed in one DATA packet and then sends out to the server. The accelerator or brake position consists of a float value between -1.0 (maximum brake) and 1.0 (maximum accelerator), the default value for brake is set to -0.44 allowing a smooth deceleration until a full stop is reached. Similarly, the accelerator value is set to 0.58 by default to limit the maximum speed in a test scenario to about 10 km/h. Another float value holds the steering wheel position with -1.0 for maximum left and 1.0 for maximum right. As a safety precaution to prevent rapid turns the steering wheel position is limited to -0.7 till 0.7. The desired gear is saved as an integer value with 1 = park, 2 = reverse and 3 = drive.
Both float values for accelerator /brake and steering and integer value for gear is packed into one 16-byte DATA packet shown in Table 1
|4 bytes||4 bytes||4 bytes|
Table 1: structure of outgoing DATA control packets
DATA packets are only sent out when at least one finger is touching the screen. If no fingers are touching the screen no DATA packets are send out, causing the car to perform an emergency stop after a short time interval. This is a safety measure to ensure that the car comes to a full stop when the connection between remote and car is lost or if the car is out of Wi-Fi range.
Communication from car to smart phone:
The car is functioning as a server for the remote and is sending back two kinds of packets: feedback and camera packets. Feedback packets are send for every incoming control packet described above, containing 16-byte information for packet version, current speed, set gear and set turn signal mode. Table 2 shows the structure of feedback packets. The speed value is displayed in the top bar whereas gear and turn signal information are used by the client to verify the last sent desired gear and turn signal commands.
|Current position||Current Speed||Gear|
|4 bytes||4 bytes||4 bytes|
Table 2: structure of incoming DATA feedback packets
Camera packets are constantly send from the car to the smart Phone. Each packet contains data for several rows of the camera image encoded with a noise filter and additional Meta information. The Meta information consists of three short values for which row is send, the maximum number of rows and how many rows are sent. The raw image data follows the Meta information as the payload of the packet. Table 3 shows the structure of camera packets.
|Row||Maximum Row||Number of Row|
|2 bytes||2 bytes||2 bytes|
Table 3: structure of incoming UDP camera packets
The remote software receives these camera packets and updates part of its OpenGL ES texture accordingly which is displayed in the main screen. With the use of a noise filter the bandwidth of camera data is reduced to one third compared to raw RGB data.
This document has described the architecture and mechanics of the smart car remote software. Completely autonomous driving as well as manual driving with a remote like a mobile phone will probably not be accessible for everyone to use in every day’s traffic for the next few years.
Nevertheless there are still areas where a car remote can be used in a beneficial way. We will briefly describe three possible applications for the future. The first is commercial use of the car remote for specific maneuvers where jurisdiction can be clearly stated. For example, instead of opening a garage door every morning to drive out the car and then closing the door again, all can be executed with a remote control right before leaving the front door. The garage door will automatically open while the car drives out before the door closes up again. This option can save time and is comfortably to use. The same can be imagined for parking in a parking space or parking structure. The second usage was already outlined in the beginning of this document: When testing for autonomous cars safeguards are required like people sitting in the car ready to hit the emergency brake in case something fails. These precautions and other actions can be done remotely with a mobile device. Mobile devices can also act as a mission control overview where checkpoints can be defined for an autonomous route. Lastly, another commercial use of mobile devices would be to let them act as a diagnostic frontend to visualize what is wrong with a car.Most vehicles today have much more diagnostic information stored than is shown on the dashboard. On-Board Diagnostics (OBD) has also been standardized for many years to refer to the car’s self-diagnostic and reporting capability. A mobile remote device can read out this information to display the necessary steps to repair the vehicle or to offer the option to call a more specialized mechanic.
We see our work on the smart Car as a proof-of-concept on which more applications can be built upon. As of future work we would like to incorporate a security protocol to our communication between car and smart phone and try to integrate more on-board diagnostic information. For a spinoff project we also envision the visualization of laser scan data and mission control for autonomous cars.
The smartly designed car, covering a huge area of its applications, it is inevitable to distance oneself from the knowledge of its future usage. After the successful accomplishment of any project, the very next step is to work on deploying its technology properly. Currently, our team of faculties and students are working on this aspect of the project. Enlisting, brainstorming and a little digging in the industry to find out the proper and apt applications of the smart car. Defense, commerce, entertainment are a few areas, out of so many others where this Smart car can prove to be assisting and also a replacement to a certain extent. It could be worth a while if this innovation can be used for the notional defense purpose. Taking this concept, to the border where our soldier patrol day and night, fight for the nation, can be really fruitful. Instead of the actual human soldier patrolling, the car – “A smart soldier”, can do the job while the man factor can be at a safe yet convenient distance. Another very apt usage of the car could lie in helping the specially abled people. The application of this technology would not stop them from driving a car. If need be, one can take a back seat and without being dependent on anyone else, they could go on, on their own. Talking about the next best application, exploring the usage of the car by taking it to people, by using it in the commercial sectors.Along with these applications, entertainment industry can also benefit from the smart car. The cars driven on screen showing stunts usually use animations, well the smart car can save you the extra expenses of very costly animations. Once our team has neatly done the research over theapplications of the car and its technologies, we would officially propose its applications in properly related sectors.
One successful project. But that’s not it for students and faculties of Amiraj College of Engineering and Technology. We at Amiraj keep going on with the innovations. Along with this project of smart car, we have several other ongoing projects. The students of different branches have come forward, clubbed together and have been trying to use their knowledge and skills for their proposed projects and ideas under the counsel of our faculties. Various projects in the field of usage of renewable energy sources, agriculture, water management and likes. Other applications by using the technology of Amiraj Dextra Smart are also being proposed.