Tag Archives: ar drone 2.0

iPhoneDrone Project – Flight Planning, Imagery Acquistion, and Ground Control

Background

No matter the scale, flight planning has always been key to any successful aerial photogrammetry/remote sensing project. The rules for UAS are no different. Additionally, a successful flight plan is characterized by knowing the specifications of the camera/sensor and having a UAS that is capable of flying with that sensor mounted.

Part of the planning process involves defining a number of specifications for the desired end-product before going into the field. Some basic considerations include:

  • Image resolution: How much detail is desired? A smaller ground sample distance (GSD or pixel resolution) will have a higher resolution compared to a larger one. The flying height and focal length of a cameras sensor determine the pixel resolution.
  • Image overlap and sidelap: Image overlap and sidelap are essential for the establishment of tie points that orient the images and create the 3D model. Traditional standards call for 60% overlap and 30% sidelap, however, UAS acquisition usually requires higher variations of both.
  • Sun angle and weather: Reducing the amount of shadows is very important when capturing aerial imagery. Having the sun at it’s peak height in the sky minimizes shadows and allows the sensor to receive as much light as possible. Wind also plays an important factor. How powerful is your UAS? How powerful of wind gusts can in handle and still maintain a level position?

So you are ready to make a successful flight and have all of the flight planning figured out. The next question, however, is how geographically accurate should it be? If one is only interested in a product that is marginally accurate, say within 5-10 meters, than the GPS from the flight log file or camera would be sufficient.

For survey grade accuracy however, having a set of survey control points measured from a professional land surveyor is necessary. Control points are the most accurate way to ground truth any georeferenced product. Land surveying is expensive, however, so a product with an accuracy within 5-10 meters may be enough for some.

Software

AOI in Red and Flight Lines in Yellow
AOI in Red and Flight Lines in Yellow

For flight planning, I use Mission Planner, sponsored by 3D Robotics. It is free to  download and very easy to use. I have an ideal ground station by combining this software and the supplied telemetry antennas that came with the 3DR IRIS. With a few simple clicks, Mission Planner is able to display flight data in real-time through a series of interchangeable gauges.

The Flight Plan tool allows the user to plan varies autonomous missions and tasks by establishing GPS waypoints. For my project, I used the survey (grid) tool. Within minutes I had my area of interest (AOI) polygon, entered my camera specifications (the iPhone 5s’ focal length is 4.12 mm) and set my overlap and sidelap (70 and 60 percent).

On top of flight planning, Mission Planner also acts as the control station for the IRIS. By using the supplied telemetry kit, my laptop was able to send and receive data in real-time to the copter. A log file was also recorded storing all of the flight parameters.

Imagery Acquisition

The desired pixel resolution for this project was 1 cm. Considering the highest resolution from traditional manned aircraft missions averages between

Image Courtesy of CameraSharp
Image Courtesy of CameraSharp

1-3 inches, resolution to this degree from a consumer grade product like an iPhone is quite remarkable.

During the flight, the camera shutter was activated using an intervalometer feature from an app called CameraSharp. The app allows you to set the seconds interval at which you wish to snap photos. For this project, I went with one photo every second. I have yet to determine the exact amount of overlap I averaged, but considering the flying speed of the IRIS was well below the necessary limit for 70%, I’m guessing I achieved an overlap somewhere in the 75-80% range. I’ll post those results in a few months when my research is complete.

iPhone 5s and Flight Specs
iPhone 5s and Flight Specs

The flying height was set at 28 m to obtain a pixel resolution slightly better than 1 cm at the sites lowest point. The figure to the right illustrates the iPhone 5s and flight specifications.

 

Ground Control

Having a professional land surveyor measure ground control points (GCPs) is the best way to accurately ground truth any geospatial data. Thankfully, I have some friends who had experience and equipment to measure GCPs for me at the Lime Ridge site. The control was measured using a Trimble real time kinematic (RTK) device. The RTK provides real-time corrections of GPS information by way of a network base station enabling the measurements to be accurate with 1 cm.

aerial Target
Aerial Target 5

Aerial targets were fabricated using some poster board that were cut 5 cm wide at the edges. Nine targets were placed around the site providing a sufficient ground truth throughout. The targets were measured during image processing.

GCP_o
GCPs overlayed on Ortho

 

Creeped Out SF Alamo Square Residents Searching For Owner Of Crash-Landed Mystery Drone

Creeped Out SF Alamo Square Residents Searching For Owner Of Crash-Landed Mystery Drone

“She says her boyfriend noticed the blinking aircraft sitting in the middle of Fulton street on Saturday night, and decided to bring it in for a closer look.”

phantom-drone
A drone, like this Phantom 2 model, was found Saturday on Fulton Street in San Francisco. (ROBYN BECK/AFP/Getty Images

Link courtesy of CBS San Francisco

iPhoneDrone Project – Lime Ridge Open Space Site

The site I chose for my project was an area of the Lime Ridge Open Space in Concord, CA. In the late 19th and early 20th centuries, limestone was mined in this area for cement production leaving behind a number of quarry ruins.  Digging stopped in 1946, and in 1974, the land was purchased through a bond measure. It is 2nd largest open space in CA.

There are two large areas of the open space separated by Ygnacio Valley Rd. This project site is just north of Treat blvd. The terrain is perfect for showcasing topography while still being small enough to acquire imagery in one flight.

Site_Pano2Site_Pano1

 

Drones Go Mainstream: Highlights from the “60 Minutes” Piece

 

A few weeks ago, “60 Minutes” dedicated an entire story titled Drones Over America highlighting the proliferation of drones in the civilian world. The juxtaposition of veteran Morley Safer (82)  interacting with a technology so virgin yet relevant made for quite an entertaining program. Overall it was an excellent showcase of the technologies growing capabilities, and true to 60 Minutes form, it featured proponents and opponents from both sides of the coin.

Below are some highlights:

  • Domestic drones are poised to become a multibillion dollar industry, revolutionizing everything from crop management to package delivery.
  • For the moment, their barely regulated
  • We’re looking at the future, and whether you like it or not, the future is looking back at us.
  • Sophisticated as they are, an idiot can fly one.
  • Guinn: “Man, I’m not really too sure what I’m going to take aerial photos or video of, but that thing is really cool, and I want one.”
  • Toscano: “We build drones for the 4 D’s: dirty, dangerous, difficult, and dull missions.”
  • The common denominator in the world of most drones is the camera.
  • Cummings: “Using a drone for pictures is no different than using high-powered binoculars or a telephoto lens.”
  • Feinstein: “When does it become stalking? When does it invade privacy?”
  • Cummings: “It’s revolutionizing the aerospace industry in a way not seen in 25 years.”
  • When will a drone be at your front door? Time and technology will wait for no one.

San Francisco Based Company “QuiQui” Offers Service for Delivering Drug Store Items Via Drones

quiquiDrone delivery is already “taking off” in the city of San Francisco. QuiQui, an SF based start-up, is offering to deliver drug store items right to your door via a drone if you live in the city’s Mission District. The FAA’s recent loss in federal court is already opening up many opportunities for would be drone companies. It’s only a matter of time before Amazon Prime Air drones are delivering your next PS4 games.

Link courtesy of QuiQui: http://quiqui.me/

What is Photogrammetry?

What is Photogrammetry?

First aerial photo. Taken by Frenchman Felix Tournachon.
First aerial photo. Taken by Frenchman Felix Tournachon.

The first aerial photograph was taken from a balloon over Paris in 1858 [1]. Following the end of World War 1, many former military photographers turned aerial photography into a legitimate commercial business by developing their own aerial survey firms [1]. Since then, aerial photography has been an important utility for mapping and surveying professionals.

Lillesand et al. (2008) defines photogrammetry as “the science and technology of obtaining spatial measurements and other geometrically reliable derived products from photographs.” Many geometrically accurate products can be produced using photogrammetric techniques. Planimetric maps, digital terrain models (DTM), and raster imagery can be photogrammetrically produced in 2D or 3D and a used for geographic analysis in a GIS [2]. The ability to accurately measure and map large tracts of land make photogrammetry a viable alternative to expensive and time-consuming field surveys.

Based on Lillesand’s description, the following photogrammetric elements are key to any photogrammetry project:

  • Image Scale: Based on flying height above ground and terrain elevation, the image scale can be used to make ground measurements relative to corresponding photo measurements.
  • Building lean is an example of object displacement
    Building lean is an example of object displacement

    Object Displacement: Unlike maps, aerial photographs do not show a true plan or top view of objects unless taken directly above the object. Object displacement causes objects, especially taller ones, to lean from their bases. Height measurements can be made, however, by factoring in the scale and amount of lean an object has. This practice is especially accurate when measuring an object or ground elevation in two overlapping images.

  • Ground Control Points (GCP): GCPs are actual measurements made on the ground by a land survey crew. They are paramount to the geometric accuracy of any photogrammetry project.
  • Flight Plan: Proper flight planning must take place to ensure the desired products can be produced in any photogrammetry project. Information about the camera, image scale, and photo overlap are all important factors in proper flight planning.

What is the Desired End Product?

Orthophotos are used in GIS as basemaps for heads-up digitizing, analysis, etc.

Ortho-photographs (orthos) are generally the desired end product generated from photogrammetry. Orthos are generated by creating a mosaic of multiple photos together that are acquired with sufficient overlap (usually ≥ 60%) [1]. Areas outside the overlap are cropped out, leaving a mosaic of nadir images. A DTM is also applied to the mosaic, which geo-rectifies each image pixel. The ortho-rectified mosaic acts not only as an image without object displacement, but also as a geo-referenced map with accurate geographic location coordinates.

Breaking Down the Process of Generating Orthophotos

There are a number of software applications designed for generating orthos. For imagery acquired from a UAS, I have found great success using Agisoft’s Photoscan Pro. The interface is incredibly user-friendly, and there are plenty of tutorials available for support.

Aerotriangulation

Image tie points and the relative orientations from the AT
Image tie points and the relative orientations from the AT

The initial step after acquisition is aerotriangulation (AT). AT is the method of orienting images to the proper geographic location (i.e., determining their position and rotations, in space). In photogrammetric jobs that only require one or two exposures, the GCPs are sufficient enough for image orientation. Jobs that require hundreds or thousands of images, however, would also require hundreds or thousands of GCPs, which would not be cost-effective. Therefore, AT bridges areas without ground control and reduces the number of ground control points needed. A pair of overlapping images can be relatively oriented to one another by measuring the exact same ground object (called a tie point) in each of the corresponding images. Photoscan automatically generates the tie points for the entire block of images during the “align photos” process. The GCPs complete the orientation of the block through methods of space resection, the process in which photocoordinates for at least four control points are measured to form a least squares solution for unknown points. Agisoft uses a bundle-adjustment to orient all of the images for a project. If GCPs are not available, GPS logs can be used for geo-referencing, albiet, with much less accuracy [1].

Auto-Correlated DTM

Digital Terrain Model
Digital Terrain Model

A reliable DTM is necessary for any orthophoto production. With an established AT solution, the software will be able to determine any point location on the ground, in relation to the images, by using collinearity equations. According to Lillesand et al., “Collinearity is the condition in which the exposure station of any photograph, any object point in the ground coordinate system, and its photographic image all lie on a straight line.” By automatically applying collinearity equations to identical points in multiple images, the software is able to triangulate the corresponding ground x, y, and z location using space intersection [1]. Photoscan has the capability of measuring millions of points automatically through this triangulation method. A dense 3D point cloud of measured points is the first derived product. Interpolating the points into a grid for the entire area will then generate the DTM.

Ortho-Mosaic

Mosaiced images of M&T Bank Stadium, Baltimore, MD.
Mosaiced images of M&T Bank Stadium, Baltimore, MD.

The final step is to generate the ortho-mosaic. Photoscan uses the AT solution and DTM to write ortho-rectified GeoTIFFs (i.e. TIFF images that have geographic information embedded into their metadata). The software applies collinearity equations to project each image pixel to its corresponding ground location on the DTM, eliminating much of the object displacement present in the raw images. Photoscan is unique in the sense that it mosaics (or stitches) all of the images together at the same time it rectifies them. The software uses an “adaptive-mosaic” method to compensate for intensity/color differences between adjacent/overlapping images during the mosaic procedure.

The other element of ortho generation is the resampling process. By resampling each raw image, using cubic convolution methods, the software creates an ortho that contain a uniform pixel size.

 

1. Lillesand, T., Keifer, R., & Chipman, J. (2008). Remote Sensing and Image Interpretation (6th ed.). New Delhi, India. John Wiley & Sons, Inc.

2. McGlone, C., Mikhail, E., Bethel, J., & Mullen, R. (2004). Manual of photogrammetry (5th ed.). Bethesda, MD. American Society of Photogrammetry and Remote Sensing.

Robot Eyes Protecting From Above: Drones and Ocean Conservation

Link Courtesy of National Geographic: http://newswatch.nationalgeographic.com/2014/03/19/robot-eyes-protecting-from-above-drones-and-ocean-conservation/

SoarOcean-600x473“Traditionally, one of the most important tools in monitoring and protection of the oceans is aerial surveillance. When used for our ocean, current approaches involve rental of flight time on private aircraft or the use of military aircraft as a secondary mission. Unfortunately, this method typically results in less desirable coverage or lower frequency of flights due to the high costs involved and pilot limitations.”

“Thankfully, there now exists a technology that can change this approach forever. The Unmanned Aerial Vehicle (commonly called a UAV, UAS, or drone) is a reusable robotic aircraft that can fly without a human pilot or crew on-board and, as a result, does not suffer the same issues associated with pilot fatigue or high operational costs.”

“Through the democratization and demilitarization of drone technology, there are a number of issues that can be solved more cheaply and effectively than our current methods.”

“Drone technology is in the midst of a technological boom. In the coming years, we will be seeing more and more uses for these platforms and acceptance into the industries that are ripe for this sort of innovation.”

UAS Applications

The burgeoning emergence of highly-capable consumer cameras combined with affordable and ready-to-fly (RTF) sUAS has empowered anyone with an interest to become an aerial photographer. The addition of youtube, geotagging, smartphones, and social networks has created a vast network of aerial video and imagery, available to anyone with an internet connection. Drone Analyst posted a great article about the Democratization of Aerial Photography highlighting this phenomenon.

Enabled by the democratization, the applications of UAS are endless. Those that receive the most attention are listed below:

  • High Resolution Aerial Survey and Mapping
1 cm resolution image acquired w/iphone and UAS
2.5 m? resolution taken from Google Earth

Being that I’ve been a photogrammetry and remote sensing professional for over 5 years, this area is of particular interest to me. The aerial survey industry has long relied on manned aircraft or satellites to acquire geospatial data. Professionals have a proven track record of manufacturing products with high levels of quality and accuracy. Three inch image resolution is the normal standard of high resolution imagery by most firms. Other sensors like LiDAR, synthetic aperture radar, and hyperspectural imagery are common acquisition platforms.

Small UAS have the potential to marginalize the current aerial survey and mapping industry. Their ability to fly low, cheap, and efficient could revolutionize the way geospatial data is collected in the next 2-5 years. Obtaining imagery with low-cost cameras/sensors at centimeter resolution is an absolute a reality at this time. What is lacking, however, is the proven quality and accuracy of these products compared to traditional industry counterparts.

  • Pipelines and Power Linespowerline_340

Utility companies are common clients for aerial survey firms. Corridor mapping is highly effective at collecting an inventory of utility infrastructure. SUAS are perfect for this application because of their ability to collect high-resolution imagery or LiDAR in only a few flight lines.

  • Precision Agriculture

ndvi farmndvi-vis-comparisonThere is a lot of buzz around using sUAS for vegetation analysis. This is achieved by acquiring imagery in the near-infrared (NIR) spectrum and then applying the normalized difference vegetation index (NDVI). Virtually any point-and-shoot can be converted to sense NIR (although quality is still unproven). The thinking around the UAS community is that agriculture will be the biggest market for NIR equipped sUAS. NDVI maps of ag fields would empower farmers to see exactly which fields are healthy and which are not. The Daily Beast wrote a great article about how drones could soon be ubiquitous to American farms.

  • Emergency Services

Search and rescue efforts throughout the world have already benefited from UAS. They are great detectors when equipped with thermal sensors. Unmanned aircraft have been used to detect and fight forest fires, and provide aid during flash floods. Whether used by public or private operators, UAS have already established their beneficial value when emergencies occur.

  • Journalism

One of the most obvious and controversial applications is journalism. It’s bad enough that the paparazzi apologetically harass celebrities on foot. Empowering them to shoot above someones private property is a whole different story. These journalism tactics strike the fear of “big brother” in many Americans.

There have been positive results to using drones for journalism, however. UAS have allowed film makers to obtain amazing aerial shots for movies, TV shows, and documentaries.   The makers of the “Flying-CAM Sarah” were recently honored by the Academy of Motion Pictures for it’s work in movies like “Skyfall,” “Oblivion,” and “Prisoners.”

  • Site Inspection

Rather than higher an expensive helicopter or manned aircraft, construction managers are increasingly turning to UAS technology. They are safer, cheaper, and provide all of the information from a bird’s-eye-view that a site manager would want to know. And they enable managers to easily monitor the progress of a site throughout the project’s life cycle.

quary demQuarry’s and landfills also benefit greatly from sUAS. 3D models can provide an accurate inventory of stockpiles and calculations of material volumes are easily attainable using sUAS acquired imagery.

  • Real Estate

Real estate is quickly becoming one of the most widely reported applications of UAS. “One 57,” Manhattan’s newest state-of-the-art condominium high-rise, used UAS as an incredibly effective marketing tool by capturing what views would look like on different floors. This enabled potential customers the opportunity to “see” what their view would be before the construction was even finished.

Some UAS operators are making legitimate profits by selling aerial videography services to real estate companies. Even though illegal, news stations from around the country are reporting on this as if it’s ok.

  • Delivery Service

1385949215000-AmazonPrimeAirOne of my favorite videos is from Domino’s in the U.K. of a drone delivering a pizza. The concept seemed far fetched initially, but in a matter months, there were videos of other companies jumping on the drone delivery bandwagon.

Amazon made the most publicized announcement when it aired a “60 Minutes” piece about future UAS delivery plans. Like many new ideas, there has been criticism and skepticism, but in the long run, I believe Amazon’s “Prime Air” will revolution parcel delivery.

FAA Regulations

logoFAAAlmost daily, I get asked “are drones illegal?” Short answer – No. The technology is obviously moving at a breakneck pace, applications are abundant and continue to grow, and the potential for big business in the United States is evident. Manufacturing start-up firms are popping up all over the country and have every legal right to make and sell as many UAS to as many customers as they please.

As of today, I would define the Federal Aviation Administration’s (FAA) current policies/regulations for UAS as  muddled and directionless. The only certainty is that UAS are not allowed for commercial purposes under any circumstance. The FAA’s UAS frequent questions website clearly defines this:

Q: “Can I fly a UAS under a COA or experimental certificate for commercial purposes?”
A: “No. Currently, there are no means to obtain an authorization for commercial UAS operations in the National Air Space (NAS).”

There are three basic categories of UAS operates defined by the FAA and how they can operate a UAS:

  • Public entities such as local governments, police departments, universities, etc. Public aircraft must obtain a Certificates of Waiver or Authorization (COA) from the FAA for approval to operate. This is a lengthy/drawn out process that takes months to materialize. In 2012, the number of COAs requested was 257. By the end of February 2013, 327 COAs had been requested in just 2 months.
  • Civil operators and private industry. Civilian operators do not have to obtain a COA. They do, however, need approval to operate by obtaining a Special Airworthiness Certificate (SAC) from the FAA. I had an email conversation with Thomas Rampulla, the FAA’s legality specialist, about the necessary steps for SAC obtainment. Although all 11 steps are rather cumbersome, they are achievable with effort. The one step that stands out though, is the necessary requirement of “being a certified pilot regardless of aircraft size or operating altitude.” This is a great barrier since 99.9% of people interested in operating UAS are not certified pilots. Furthermore, I stated to Mr. Rampulla that the aircraft I wanted to operate weighed less than 3 lbs and posed little to no danger to the public. He balked at my statement, said it didn’t matter, and said that I needed to be a certified pilot no matter what. Needless to say, this was a very frustrating conversation.
  • Hobbyist make up the third category of UAS operators. Any person operating a UAS for recreation or sport falls into the hobbyist category. Hobbyist are to abide by the 1981 document AC 91-57. Hobbyists are exempt from any certification or approval from the FAA and AC 91-57 is their only authority. Basic standards of AC 91-57 are: operation away from populated areas, maximum flying height of 400 ft above ground level,  remain within line-of-sight, vehicle must be <55 bls.

To sum things up, it is very difficult to get proper clearance from the FAA if you want to fly for any reason other than recreational use. My initial reason for contacting Mr. Rampulla was to inquire about using my IRIS for an R&D at Towill. I was quickly shot-down by him with little to no explanation as to why a 3 lb UAS needed a certified pilot for R&D but not for hobby use.  As I’ve come to learn, most UAS operators scoff at the FAA and its inability to write sensible legislation. This leaves most UAS operates in a middle limbo-zone where they fly small UAS (sUAS) under hobbyist guidelines, but are doing work and research far beyond anything that would be considered recreational. At the same time, the FAA would have a hard time prosecuting these folks because they ultimately could come back and tell a judge that they are flying for the “fun of it.” In this 2007 document, the FAA acknowledged the necessity of having another class of UAS operators who would fall into this limbo category. Seven years later, nevertheless, UAS technology has exploded at a momentous rate, and the FAA has done absolutely nothing.

Everything is supposed to change in September 2015 when the FAA is mandated to have it’s UAS rule-book completed. Hopefully. There have been numerous delays to progress already, however, even though the FAA has acknowledged the problem. For the sake of this multi-billion dollar industry, let’s pray they get their shit together on time.