Surveyors and Other Professional Drone Operators Should be Focused on Accuracy and Reliability

Lately, there’s been a lot of talk about what it can and does mean to utilize a “professional” drone for commercial projects as opposed to a “consumer-grade” product. Different people define these terms in different ways, but for most, the crux of the argument comes down to a debate about “DJI vs. high-end”.

In practice, though, it’s often more about “phantom vs. custom.” The DJI M600 and M200 series face a lot of the same challenges as boutique manufactured drones from other manufacturers. Many of the models in DJI’s “professional” or “enterprise” lines do not, in fact, produce better accuracy when used for surveying, and since they are more expensive and more complex to operate, they do not make as good an investment for most survey companies.

This is the kind of ROI-focused decision-making we’ve helped hundreds of surveyor work through in order to use drones to grow their business without hiring, which is for the most part possible on account of providing land surveyors with everything they need to use drones to finish surveys 90% faster. Aerotas does not have any formal affiliation with any particular drone company, which gives us the freedom to test and evaluate everything in order to provide our customers with the best tools. In our testing, we have found that there are minimum technical standards that are required to produce the accuracy land surveyors need: a 20-megapixel camera with a global shutter (as opposed to a linear rolling shutter) on a 3-axis gimbal on a multi-rotor. The DJI Phantom 4 Pro and Advanced models meet these specifications. With the right sensor, so do the more expensive models, but those more expensive models do not produce better accuracy results on surveys.

There’s definitely some skepticism when we talk about these concepts and numbers, especially when we’re able to do so because we’re using what some people dismiss as “toys”. The problem with this mentality is that it doesn’t recognize the psychological bias we are all subject to which leads us to associate quality with price. We think that just because something is less expensive, it must be lower quality, and the more expensive option must be the higher quality one. This bias is called the Veblen Effect, and it’s something that is not limited to the drone industry.

We often see this play out with land surveyors and engineers when they first approach us, who think that because they’re professionals, they need a drone that is marketed as “professional grade.” It’s why we coach land surveyors and engineers to instead focus on what variables are most important to them, in order to avoid this trap.

The most important variable for drone-users in these professions is the ground-truthed accuracy of the data produced. Second to this is reliability, and finally all-in price. This is the analytical approach we took to determine which drone is the best tool for surveyors, and it led us to the DJI Phantom 4 Professional. Whether you want to call it a “toy” or anything else doesn’t really matter to us, because it produces the best results.

I do actually agree with some of the arguments for using more “professional” grade drones in certain applications, but those applications are not relevant to most surveyors. I absolutely agree that that data security in the context of evidence gathering by police or work around infrastructure critical to national security needs to be a priority. How often is your survey going to be taking place in areas like this though? If you’re conducting a survey at or near an area Homeland Security has identified as being important, this issue should be a top priority. For a topographic map of a subdivision outside of Reno, though, it may not be so critical.

The importance of professional Standard Operating Procedures is also something I hammer on every time I give a talk or seminar, and it’s a primary focus of the hands-on training we do for our clients. SOPs specific to the type of drone operation (high-accuracy surveying, for our customers) are the only way to ensure you are getting repeatable results, and doing so safely, legally, and profitably. It seems like this is starting to gain real traction to the point that we now often have new customers come to us specifically because they want to be trained on the SOPs we’ve developed. We’ve actually been working with the National Society of Professional Surveyors as they’re discussing a drone certification of some sort specifically for licensed surveyors.

For the type of work most surveyors do, prioritizing a drone that has redundant onboard systems in order to avoid “single point of failure” is not a critical concern. Our customers would rather use a drone that’s so reliable that it doesn’t need redundant systems. The Phantom is produced in staggering numbers by comparison to any other drone, which means the engineers at DJI have had the opportunity to work out the 1 in 10,000 type issues and kinks in a way that high-end boutique manufacturers just don’t have the opportunity to do. Across the hundreds of surveyors we’ve trained and set up, I think we’ve only had one or two issues with a Phantom.

The beauty of DJI drones, particularly the Phantom series, is that they are closed systems designed to work together, so they are incredibly reliable. Whenever you start customizing, you are invariably adding complexity, which is going to lead to problems with field operations, reliability, and data. When you start integrating third-party systems, there is going to be a much higher frequency of things going wrong. We hear examples of this from clients regularly. One surveyor we spoke with recently purchased a $70,000 Trimble drone a year ago and has only successfully used it on two projects because of constant issues with the gimbal and flight software. Another time recently we did an accuracy test bake-off between a Phantom (used with Aerotas SOPs) and an M600 with a Canon DSLR camera used by a local premier engineering firm. we finished the entire operation in under 30 minutes and wound up with data with better than 0.1’ accuracy, whereas the team with the M600 wound up having to come back 4 times over a few weeks before they successfully completed the mission, and when the did half of the photos taken were inexplicably blurred. And the folks operating that M600 are very proficient and experienced with drone technology. We prefer equipment that just works.

The minimum technical standards we’ve determined in testing means many “lower” models do not produce equivalent accuracy that we see from the Phantom series, but there are still occasional applications for those models. We do have some clients who we have equipped with the Mavic because of its portability and simplicity, and because they do not need survey-grade accuracy for some of the work they do.

The needs and requirements of the project should be driving the choices around what technology is being utilized, rather than arbitrary distinctions as what qualifies as “professional” or “consumer-grade”. Anyone solely focused on only using the former needs to challenge their assumptions about what these terms really mean. Often the only thing defining a drone as “professional” is that the manufacturer advertises it as such. Rather than relying on marketing and advertising, rely instead on the accuracy and reliability that the tool you’re using can provide.

 

 

 

Tags: , ,

About the Author

Daniel Katz is co-founder of Aerotas, where he has helped hundreds of surveyors nationwide start surveying by drone with industry-leading accuracy. Daniel regularly speaks at survey conference across the country, and is a regular contributor to Point of Beginning and state survey association magazines.

10 Responses

  1. Mark R. Culpepper

    We’ve flown thousands of acres of projects and in general, the issue of accuracy generally has less to do with the drone and more to do with the sensor. While I generally agree that DJI has done an impressive job integrating a very good sensor (not a great sensor) with a well designed piece of hardware. The problems with DJI are three fold; culturally they are a Chinese firm beholden to the Chinese government with all it’s associated baggage – which is not insubstantial and cannot be addressed with a firmware upgrade. It is telling that DJI’s first geo-fenced area was Tienanmen Square; they dictate where you can or cannot fly which is a bit like Porsche telling me where I can or cannot drive my car and at what speed. Second, and related to that first point, is that despite DJI’s claims to the contrary anyone who uses a DJI is essentially creating a remote sensor for the Chinese MSS. I have neither the time nor the inclination to mentally check myself at ever job site to ask “is this site relevant to the security of my country?”. I shouldn’t even have to ask that questions. Third, they do not understand enterprise software and the issues associated with random and intrusive upgrades. I literally cannot tell you how many job sites we were unable to fly because of an unannounced firmware upgrade or a flight restriction in an area that required neither new firmware nor flight restrictions. Where I fly is up to me and my pilot operators; that’s why we are all licensed and insured. Reliability means more than just the hardware. DJI’s problems are structural, job killing problems and what ultimately led us to go down a different path and use a different vendor. We’ll pay the premium to make sure we have a reliable high grade sensor without Big Brother telling us where we can or cannot fly, who sends the data back to a foreign governments intel agency. The alternative is to put my job and business decisions in the hands of Chinese manufacturer who has absolutely no idea what it is that I do.

    Reply
  2. Michael Boland

    +1 on Marks comments.
    DJI, despite their claims, are a hobby item.
    Would you run a trucking business with trucks made in China that can only be serviced in China and spares only come from China.
    I design and build commercial drones and I see a lot professional tearing their hair out trying to get contracts because some ex-butcher or ex-electrician has bought a hobby drone and some software and under bids on major contracts.
    We all know the result but the clients don’t really appreciate the impact of error.

    As an example, I was called in on a new contract to do the flying over stockpiles of coal.
    We were only 1000m away from an airport and they were long flights.
    The first results showed the company a 40,000tonne shortfall in stock volume.
    They immediately pointed the finger at the figures from the drone being wrong, naturally, as they had professional surveyors in doing the stockpiles bimonthly.
    So the second survey flight was lucky enough to have one stockpile untouched from the first.
    It shoed a 0.3% reduction in volume from the first survey a month earlier. (settling)

    So any companies out there taking the lowest bidder are putting their neck and future on the chopping block.

    As I tell these professionals when they are crying on my shoulder about underbids, just smile and nod, smile and nod.
    The companies that survive will be the ones that recognise accuracy and consistency in this area.

    Mike

    Reply
  3. Noah Ruiz

    Mr. Culpepper is right. Though DJI has built several robust systems, they are putting a bad taste in peoples mouth for what we are trying to do for the industry. There are several platforms out there that have great sensors on them, which differentiate accurate data to low quality data. Something to take in account for the use of DJI’s Phantom Series, it is labeled on the website as “consumer” grade products, which are for “consumer” use. Fancy Pictures. As you stated ” In our testing, we have found that there are minimum technical standards that are required to produce the accuracy land surveyors need: a 20-megapixel camera with a global shutter” which only one DJI Phantom Series honors this and is Consumer grade, still does not give you a great sensor but a good sensor as Mark stated. Half the battle in this industry is educating people on the difference between consumer grade products and professional grade. it is kind of hard to tell someone to use a DJI Phantom as professional equipment when it is listed as Consumer grade on their website. No need to mention ALL the issues that Mark mentioned before. They have a purpose and the purpose is not for providing accurate data for Land Surveyors, but taking sweet photos.

    Reply
  4. Lance

    So which path have you taken? I have some similar opinions but need that quality alternative.

    Reply
  5. J Keith Maxwell

    Great article Daniel. This is one of the best to talk about the “problems” or issues with Drone use and offer suggestions. I like your minimum technical components for a usable machine for surveyors. I also like the SOP’s as well and I’m sure they’ve been developed over time. I’ve personally been concerned with the number of surveyors getting into this space, with very little knowledge, and claiming extremely accurate mapping that they cannot actually produce. The cost of entry has been a big cause of this.

    I think you’re on the right track with developing some standards for all surveyors producing this type service to use and adhere to.

    I’m interested in what Mark has to say about DJI but it’s hard to actually believe every conspiracy theory out there. We’ve used tons of foreign equipment over the years in our profession. Wild-Heerbrug (now Leica) was one of the first for me. But that didn’t have any radio capabilities to send info back to the motherland. Today we have Topcon, Leica, Zeiss and a slew of others foreign companies producing surveying equipment. As I see it, normally, the biggest problem from China is quality, and possibly toxicity (chinese drywall and lead based paint on toys, etc).

    Reply
  6. Sean David Goertz

    I wouldn’t personally dismiss what the Homeland Security says as merely a conspiracy theory

    https://www.engadget.com/2017/11/30/homeland-security-claims-dji-drones-spying-china/

    Using equipment manufactured in another country isn’t the problem. The problem is with data being accessed by an unauthorized 3rd party or foreign entity. A pilot might not care, but the client and end-user of the data might, and that’s what really matters. That does not look good on the pilot if they choose to dismiss that concern if the client cares about data security.

    I’m concerned with the number of individuals getting into this space, with very little knowledge, and claiming extremely accurate mapping that they cannot actually produce too. A vast majority of the time it’s because they’re using something you can buy at Walmart rather than certified equipment acquired by a licensed positioning equipment distributor, like Topcon.

    Reply
  7. Tyler Dears

    36.4mp > 20mp. This is by no means a “psychological bias” but rather, an objective fact. Higher sensor specifications will yield higher results, provided proper sensor operation is observed. As well, the aviation industry is founded on safety. The importance of safety need not be emphasized any further but, we as UAS operators should strive to constantly improve our practices. At the end of the day, operators rely on the UAV to execute the commands of the PIC. In a perfect scenario, no accidents would occur because of a system failure. Redundancy within the entire unmanned aerial system is key to achieving this goal. The stigma related to the UAV community is one of the largest concerns that currently faces the industry. Triple redundant IMU, compass, and GNSS sensors coupled with motor and battery redundancy is for the betterment of the industry. These items help to ensure that in the case of the UAV experiencing a catastrophic failure, it is still able to be recovered safely. The use of professional grade equipment does not come down to a “physiological bias” of a higher price. Professional grade equipment should be used in professional situations, hobby grade equipment should be used in hobbyist situations.

    Reply
  8. Daniel Katz

    Great points and comments all around. I’m encouraged by the robust debate!

    DHS concerns should absolutely not be disregarded, but for the vast majority of standard land survey work, DHS expresses no concern.

    We have hundreds of clients flying Phantoms and have had only 2 issues, both of which were due to pilot error. Regardless of what section of their website DJI categorizes the Phantom, that is unmistakably professional grade performance. We have processed thousands of projects for our clients from Phantoms, and the accuracy is consistently stellar — 0.1′ or better when following SOPs. Larger sensors allow for equivalent accuracy at higher altitudes, but do not provide better than the 0.1′ accuracy the Phantom can produce, because of the introduction of substantially more warping, artifacting, and noise. More data is not always better.

    To be clear, we have no affiliation with DJI and would really like a viable alternative that can compete on cost, reliability, and accuracy. To date, no other manufacturer can get close.

    Ultimately the drone and the sensor are important, but far from the most important factor in getting good data. What matters much, much more is having well-developed Standard Operating Procedures for ground control, mission planning, and data collection; and having experienced operators managing and running QA and QC on the data processing. We’ve produced better accuracy data with a Mavic than we’ve seen others produce with Reigl’s $500k LiDAR octocopter. The drone is just a flying camera.

    Reply
  9. Ty Brady

    I’m with Daniel on this topic. The DJI Phantom 4 Pro drone has a great camera integrated into the drone that creates a very accurate and stable system. I have done several experiments and the results agree, you can get consistent accuracies of 0.1′ and less.
    My concern would be drone operators who do not use accurate ground control points with survey grade coordinates or drone operators that create topographic mapping for design that are not licensed surveyors.
    Why do I need a $10,000 drone when a $1,500 one will produce equal results? If I crash a $1,500 drone, I can absorb that. Not the same with a $10,000 drone.

    Reply
  10. Lewis Graham

    A couple of clarifications.
    The P4P, X4s and a few of the X7 lens employ a mechanical shutter, not a global shutter. This is an important distinction since the mechanical shutter must be enabled and functions only below a certain (but certainly adequate for mapping) shutter speed.
    We have done extensive camera testing here at AirGon/GeoCue (and our team is part of the group that built the Z/I Imaging DMC mapping camera so we kknow a bit about photogrammetric cameras) with a focus (pun intended) on what is important for photogrammetric mapping – mechanical (or global but these are rare in large pixel count CMOS sensors), ability to calibrate the camera and stability of that calibration over a reasonable period of time.

    Surprisingly the DJI cameras with mechanical shutter and used in mechanical shutter mode meet these criteria extremely well. You can easily do high accuracy mapping with a Phantom 4 Por (but not the prior generation Phantom) with no problem whatsoever. Of course, you really have to know what you are doing….. That is another story entirely!

    On the subject of Chinese companies using their technology to send information home – I don’t know about you guys but I don’t think we have ever mapped anything for which someone other than the client would have the faintest interest. If you are worried about insidious Chinese technology, the first thing you need to ditch is your iPhone!!

    Reply
Leave a Comment

Leave a Reply

Your email address will not be published. Required fields are marked *