Hello to all readers of Geektimes. Today we have a chance to chat about real development of own ideas with Andrii Seleznov, co-founder of startup Petiole. We will try to avoid winning relations and suggestions how it is easy (or difficult) and enjoyable to feel media exposure on yourself. 

Instead of this, we want to write about quite a lengthy process: about how a new idea is emerging, about theoretical underpinning of author(s) knowledge, about how an idea gains its meaningful characters, and foremost, how it appears and  imposes the auditory. We mean the honest story about finding investors and further development.

We would like to avoid any advertisements but should say a couple of words. What is meant here is technically special-purpose application Petiole, that allows measuring leaf area of plants and chlorophyll content with the help of a camera of a mobile phone.

Let’s start from the very first jump. At the first sight, Petiole has a very specific task. How have you come up with the idea to get this problem? What is the potential auditory for this application?

Sure, Petiole addresses a quite specific problem. And this is our peculiarity. The idea had appeared when the problem had lived with me for three months. To be more exact, I had shared a room with my friend Vyatcheslav Bykov. At hat time he was working at the Institute of Evolutionary Ecology of the National Academy of Science of Ukraine. He was researching the influence of negative factors on Lombardy Poplar (Populus pyramidalis). For this task, he had measured leaf area. The process looked like budget-friendly and not in the optimum way — a scanner, Photoshop, ImageJ via command line, tabulating data in Excel. Said that licensed software is used in the mode of “endless demo-version” and this had added salt to this type of research. Vyacheslav had come home late, having green fingers because of permanent contact with dried and fresh leaves. I had wanted to help him and in one evening tried to be clever that this task can be solved using a simple mobile phone. One thing led to another, and things were off and rolling, and finally  we succeeded and realized this. Potential auditory for our application is  у приложения various. First of all, it is biologists, ecologists and other scientists who research plant morphology and plant physiology. Then we have become interesting for scientists in agrochemistry. After that, we have added chlorophyll measurement and started to receive calls from the big agrarian companies.

Photo credit: Petiole

However, in numbers, our performance differs from the auditory of popular social networks. Now we have 88 active users every month. The growth is not very large from month to month but we have it and it is our small reason to be happy.

OK, so hang on the “this task can be solved using a simple mobile phone” and “One thing led to another, and things were off and rolling, and finally  we succeeded and realized this”. Tell us, please, about your experience in programming and how much time have it taken to make the working code of Petiole?

Programming experience is different and quite exotic. I had written my first computer program when I was nine, using the programming language FoxPro 2.5. My parents worked at the Computation Centre of Donetsk Railway. After school, I liked to visit them and look at fat books with source code. Usually, I covered them with my drawings. After this, I had played with Pascal, Delphi, Qt, Ruby on Rails. I want to mention that I do not have the classical computer education; in my diploma, I am an engineer – a spatial planner. And at the “galleys” (it is a jargon name of outsourcing IT-companies in Ukraine) I have worked not more than two years. But this does not hinder me and my young team to write a working code for Petiole in approximately seven months (with breaks). Taking into account that we have rewritten the basic algorithm three times.

As the process progresses, was it clear that the project will have its future? Have you launched blog, were you seeking links to the investors, did you contact with your target auditory?

No, approximately until the half of the way, the project was aimed to solve the problem of one person.  But in June last year we became a finalist at GIST Tech-I. And from this moment, the project has acquired the global tasks and purposes. The blog was created three months ago. As for investors – we actively contacted with them in autumn last year. And we try to keep constant contact with our target auditory and receive feedback from them. Mainly, our users are Ukrainian scientists.

Can you tell more about GISTech-I. Sure, it is possible to google everything but we are interested in your personal attitude to this. Why is it beneficial, cool, useful? By the way, how big is your team?

GIST Tech-I is a global competition in technical and scientific projects for developing countries. It is organized with the support of U.S. Department of State and is implemented by AAAS (American Association for the Advancement of Science). Superlative of the competition takes a course as part of the Global Entrepreneurship Summit. Besides the point, we have become the first finalists from Ukraine for all the time of this competition from 2011. The main difference of this event from the other competitions is a participation of the teams only from the developing countries. Projects were assessed by the  criteria of orientation to macroeconomic indexes for society and their improvement. Personally, the semifinal experience was priceless for us. It required gathering as many votes for our project as possible during the on-line voting for one month. The main point of this voting is that one person can vote every day.

We got 5775 votes and this helped us to win 15th place in the category “Ideas” and go to the final. The final consisted of two days training (how to establish a company, marketing, promotion in social networks) and three days of participation in the Summit with a pitch of the project on the big stage. Personally, I have obtained many impressions about Africa as well as about the summit and the competition. The main point – I have met great people from Africa, Mexico, Kazakhstan, Azerbaijan, Malaysia, Chile, Peru, broaden horizons and the first time in my life – flew in planes.

We have 8 — 10 people in our team, who greatly help us in different stages of project formation. Unfortunately, we do not have a budget to pay salary. But despite this people have worked in their spare time and for our idea. Mainly, they are my friends from the scholarship program “Zavtra.UA”.

Where did this event take place? How was it? With whom have you been in competition? And how have you appeared on this list?

The event took place in Nairobi – the capital of Kenya. The first two days of the training were hosted by Conference Hall of the 5-stars hotel, Sarova Stanley. Other three days – at the territory of The United Nations Office. In general, a work environment was pretty good. Every finalist had a target to win. Constant training of pitch and preparations of the presentation till 3.00 AM. Our project entered in competition within the section “Ideas”. I believe that all 15 participants were well-matched. The rivalry was quite strong, and my weak speaking English a little bit decreased my chances for a winning. Major awards were obtained by the teams from Africa and South America.

We have been selected to the list according to results of “Startup Open 2015”. This is a global startup competition, including developed countries. It was an unexpected pleasure for us. And I am proud, that our project was listed among such startups like Lishot, AeroAnalytics, AppleDoc, BethClip and Smart Mobile Farming.

ОК, so let’s come back to the practical side of the question. The application existed even before you took part in the GIST Tech-I. After this, you participated in different events. What has been changed? On the contrary: what feedback are you receiving now? From the scientists or agronomists?

Sure, the working application existed even before we participated in GIST Tech-I. But, firstly, we used an algorithm with calibration square. It was providing not accurate measurements. We often experienced a systematic error in two square centimeters. Plus instability of the application in general. Also, we did not implement a function of chlorophyll content measurement using a dark green colour index. Secondly – in sprint 2015, an active development of the application was postponed. Generally, I forgot to tell that we also have a data cloud for collecting measurements of all users. But this is a separate story.

Then we had a prototype with basic functionality. In autumn 2015 and in spring 2016 we took part in approximately ten events (conferences, fairs, exhibitions, picnics), where we had told, shown how the app works and what is possible to do with this. Above all, we had  demonstrations at the “Scientific Picnics” in Kiev. In the narrowest sense, there we had our first public demonstration in summer 2014. The main change for us – people have started to understand that a smartphone could be used not only for games but for solving difficult tasks. We got requests from the different universities. Even we won the third-place award at the first Ukrainian agrohackathon (in Ternopil).

Now we have greater feedback from the scientists. Recently I have got a mail from our customer in the Ternopil National Pedagogical University, Department of Biology and Chemistry. They have tested Petiole with a fine-tooth comb in the context of leaf area measurement. The result – an area statistically is correct, measurements are conducting in a convenient and quick way for users.

From the agronomists, we will receive a feedback but only in 3016. The main reason is that generally agronomists are not interested in our application. They suggest a competitor in it. Our client is an owner of agribusiness, who would like to make the agronomist’s work automatic and completely exclude him from the decision chain. But our application in existing form – to go through the fields with an app – is not particularly interesting for them (in Ukraine). The total area of an average agriculture company is starting from 10 000 hectares. If we consider South Korea, Petiole is an ideal solution for them since an average area of a farm is 2 hectares.

As it is clear from the photos, additionally to the application your user must have a calibration pad or something like that? Can you tell us the details? How much does this cost? Does it influence on the final result of using Petiole ?

The calibration pad is a necessary and honey-drop addition to the mobile application. In our marketing materials I always forget to explain that it is essential only for leaf area measurement. For measurement of chlorophyll content it is not required at all. Why have we decided to make the calibration pad for area measurement? It helps to decrease complexity of an algorithm in many times. And also it increases overall performance. In my presentations I like to say in my presentations that having the stand you can do three things with two hands. Keep smart-phone, keep a plant leaf and press the buttons on the screen of the smart-phone. The first version was made using aluminium. Quite expensive but  it was possible to change the height of the stand.

Experienced folks advised me casting plastics. But after hearing the production price of the shape for casting, we decided to wait a little bit with this idea. Then, by chance, I  went to the big construction retail shop and had seen steps of beech and anti-glare sheets of polystyrene. And understood that this is what I need. All of these materials are cutting on a frazer, and is united with velcros.

Production cost of this stand is our commercial secret. Selling price — thirty (30) USD. For academic use we provide stands free of charge. Eight stands have already gone. Sure, its availability influences on the decision to use Petiole. All we dream simply to put a smartphone’s camera to a plant and get just about the complete chemical makeup. But if a person want to solve this problem, there are some hidden ways. We also help with this and offer to download a file with check-board and instead of special stand to use a stack of books. Two weeks ago our app was reviewed by Greenappsandweb. Using this method they could make the whole cycle of measurements without a stand.

Ok. Broadly speaking, what is the algorithm of the mobile app? What libraries are in use? What problems have you had and what sources or consultants had you used?

On Android-level we use OpenCV library for computer vision. In the application at this moment we use two algorithms. The first is measuring leaf area. The second is defining dark-green index. For the leaf area we are checking the matrix of homography between the  flat white area of receiving pad and matrix flat area of a smartphone. The process is very similar to calibration of camera lenses with chess-board calibration pad. Then we process the image from the camera (a leaf on the white background) with the purpose to get a good outline.

The coordinates of outline points are recalculating from the camera’s coordinates to the real ones. Using this meanings, we find a leaf area. Simultaneously, we work with a shade minimizing it and with petiole, cutting it out. For dark green index we change the modes from RGB to HSV. Then we divide channels and using the colour grade, we find the average value for the leaf within the required range. After that – we normalize it to the range and get index. Using the dependencies between the chlorophyll and dark green index (in general, linear analogs like y = ax + b), obtained in our agrilab, we re-calculate chlorophyll content directly in the app. Additionally, we use Fabric, Volley, Material Design.

Screenshot of Petiole, an app to measure individual leaf area and total plant leaf area

On early stage we got problems with accuracy of leaf area measurement but we have solved them after adding search of the matrix of homography into the measurement process. After that we had some issues with productivity. Unfortunately, smartphone is not the same as a computer; in general it has quite weak processor. Google will solve this problem simply sending data into the cloud. But there are places with no Internet coverage, 3G and other delights of civilization. Particularly, in field within the experimental plots. Algorithm worked slowly and had been broken many times with errors like the full memory void*cv::OutOfMemoryError(std::size_t). However, after re-writing it for the third time, we solved this problem and changed its strange behaviour.

Regarding resources – we got great help from the e-books on OpenCV (Packt Publishing), knowledge base of answers.opencv.org, public repositories on Github and strong documentation exactly in the website of OpenCV library. Sure, we needed knowledge in Android. In particular, for setting up compilation of native algorithms in libraries, using NDK for a smooth work. But we have filled this gap thanks to amazing resource Udacity.

In other words, a part of the code is written not in Java, but using С++ ?

Yes. Code for work of the app, saving data in the database, has been written in Java within Android SDK. Code for work of the algorithm has been written in C++ with Android NDK transformed into static library. At the start, the app is uploading the library and uses native calls for functions of OpenCV library. In general, there is Java-library for using OpenCV directly in the code of app, without JNI. However, productivity is different comparing with native calls of code. Now we have productivity approximately 5FPS when measuring leaf area in real time. Using java-library I suggest something near 2FPS.

Thank you for answers with technical details. Now I suggest to systematize. Can you briefly make the list of steps for guys who do not have professional support. But they would like to show off their technical solution of a real problem. In the way to let potential investors know about them.

It is quite tough question. But let’s try. First of all, we need to understand what does “professional support” mean. If we are talking about money – the best way to start is just to earn them in full-time or freelance job before starting the whole startup story. If we are talking about kind of protection – it is a pure luck. Suggesting media coverage, it is very important to have a good network of journalists (but liver will be harmed) or good media subject (ie, win at the hackathon). Having at least one but in the best cases – all of these options, will increase chances to be known by potential investors. However, we have one small “but”. To be known does not mean to be invested in. Almost all potential Ukrainian investors know about our project. But they do not invest any money in it. Why? This is a philosophical question.

Author: Evgeniy Shmorgun
Original text: Geektimes