Aware Group: Artificial Intelligence consulting

Over breakfast recently I had the opportunity to talk to Brandon Hutcheson,  CEO of the Aware Group. Brandon told me their 17 person company is the fastest growing artificial intelligence consulting group in New Zealand. He shared a little of their history and plans.

Brandon is a serial entrepreneur, having being involved in two successful exits with Cheaphost and Dvelop IT.  The Aware Group got their start in 2016 after successfully winning the Microsoft Excellence Award in Technology Delivery for a tertiary business intelligence implementation. This includes predicting things like student drop out rates (a task Jade had told me they have been working on too) , student prediction,  and  measuring tutor performance. This platform is now being used by various New Zealand tertiary institutions.

They’ve developed their machine vision capability across various projects, including counting and classifying traffic (cars, trucks, vans, bikes, etc) for Local and National Government.  They’ve also put this technology to use to count students going into tertiary classes. The next stage for their product release is the use of facial recognition to identify who is in the class but they don’t feel that we are socially ready for this level of implementation.

Aware group people counting

Their growing data science team is not reinventing the wheel. Where possible they’re using existing models and tweaking them to meet the needs of their project. They’re using Microsoft technologies and are a strong Microsoft partner.

The newest AI capability they’ve developed has been around natural language processing. This has been supported by their recent entry into the Vodafone Xone incubator. They’re applying this technology to understand customer support calls and attempting to surface relevant articles from an internal knowledge base. These articles could help to answer the support call more efficiently, but also to up-sell the customer on relevant services more effectively. This product is currently under development and is being tested in a call centre.

Probably some of their highest profile work has been in demonstrating AI concepts, often in partnership with Microsoft. Some of these are quite quirky, including:

  • demonstrating how patrons with empty glasses can be identified so bar staff can come and give them a top up
  • allowing conference attendees to have a coffee of their choice ordered as soon as they are identified on a camera
  • Using a recurrent neural network to generate Simpsons lines. This won the Just for fun category at the Microsoft Data Insights Summit

aware group simpsons

On the business side, the Aware group are getting most of their revenue from service contracts with an increasing growth in product-based revenue. They have bootstrapped to date, but are preparing themselves for a capital raise.

One of the challenges Brandon is facing is finding good quality AI practitioners.  As well as their team of 12 in Hamilton, they have 5 people based in Seattle and are planning to expand in the US market. They are also eyeing opportunities in Australia, Japan, Israel, and Korea.. With customers in sectors as varied as education, government, large corporates and agriculture, my main advice to Brandon was to focus their small company geographically and within a single sector so they can benefit from getting known within a niche.

The Aware Group was recently acknowledged as a rising star by Deloitte as part of their Fast 50 awards. I’ll be following to see how far and fast their star does rise and wish them all the best.

 

Lincoln Agritech: using machine vision for estimating crop yield

I recently had the opportunity to visit Lincoln Agritech where I met with CEO, Peter Barrowclough, Chief Scientist Ian Woodhead and their machine vision team of Jaco Fourie, Chris Bateman and Jeffrey Hsiao. Lincoln Agritech is an independent R&D provider to the private sector and government employing 50 scientists, engineers and software developers. It is 100% owned by Lincoln University, but distinct from their research and commercialisation office.

Lincoln Agritech have taken a different approach to developing AI capability. Rather than hiring deep learning experts, they have invested in upskilling existing staff by allowing Jaco, Chris and Jeffrey to take a Udacity course in machine vision using deep learning. The investment is in the time of their staff. Having three of them take the course together means that they can learn off each other.

The core projects they are working on involve estimating the yield of grape and apple crops based on photos and microwave images. The business proposition for this is to provide information for better planning for the owners of the crops, both in-field and in-market. Operators of these vineyards and orchards can get a pretty good overall crop estimate based on historical performance and weather information. However, they can’t easily get plant by by plant estimates. To do this they need an experienced person to walk the fields and make a judgement call. A machine vision based approach can be more efficient with better accuracy.

The team elected to tackle the problem initially using photos. They had to take the images carefully at a prescribed distance from the crop, using HDR (this is where you combine light, medium and dark images to bring out the detail in the shadowy canopy). Like most machine learning tasks the biggest problem was getting a tagged data set. The tagging involved drawing polygons around the fruit in the images, including fruit partially occluded by leaves. There was a lot of work trying to train people to do this properly. Inevitably at some stage there were guys with PhDs drawing shapes, such is the glamour of data science. This problem is similar to that faced by Orbica who built a model to draw polygons around buildings and rivers from aerial photography.

In this image labels of a fixed size are added to an image to tell the model where the grape bunches are.
This image shows the result of a trained network automatically finding the areas in the image where the grape bunches are.

They used a convolution neural network to tackle this problem. Rather than train a network from scratch and come up with their own architecture, they used the image net winning inception architecture and adapted that. This network was already trained to extract the features from an image that are required to classify 1000 different classes of images. This technique is called transfer learning. This model works well, with 90% accuracy (on their validation data set).

However, part of the challenge here is that the images do not show all of the fruit that is on the plant. The only way to get the “ground truth” is to have someone go under the canopy and count all the fruit by hand. This is where the microwave technology comes into play.

The company is recognised internationally for their microwave technology in other projects. The way it works is a microwave transmitter emits microwaves and then detects the reflections. The microwaves will travel through leaves, but will be reflected by the water content in reasonably mature fruit.

The machine vision team is working to create a model that can use the microwave image and photo to get superior performance. This is a harder problem because this type of sensor fusing is less common than regular image processing.

The team is using the Tensorflow and Keras platforms on high end machines with Nvidia Titan GPUs. There were a few raised eyebrows from the company when the team were asking for what essentially looked like high end gaming machines.

I applaud Lincoln Agritech for investing in their deep learning capability. The experience they will have gained from their first projects will make each subsequent project easier to apply the technology. The fact they have three people working on this, provides redundancy and the ability to learn off each other. This is a model that other New Zealand organisations should consider, particularly if they’ve having problems finding data scientists. Applying the latest AI technologies to agriculture seems like a real opportunity for New Zealand.