A few weeks ago, a mobile game called “Dream of Home Country” was very popular. After playing on my phone for a day or two, I found two problems:

1. Receiving gold coins is too boring, you only need to slide your finger on a fixed route, which will be annoying for a while

< p>2. I can’t catch up with people who are quick to catch up (of course, it also requires a certain strategy)

When I first played this game, I was still ranked first among my friends, but it fell after three or four days. In the third place, it is really no match for young people. . .

Just recently I was studying the things about Vision in LabVIEW, and I was wondering if I could write a program that automatically collects goods and collects gold coins, so that I can run the program before going to bed at night. You can get a lot of red envelopes when you wake up in the morning. Just do it!

Frame design

The overall idea is to use image recognition algorithms to identify what goods are on the train, and then transport the goods to the corresponding building.

After researching the screenshots, it is found that the game has the following characteristics:

1. The longest stay of the small train is about 90s each time, and it can transport up to 3 kinds of goods each time, each kind of goods The quantity is generally less than 5

2. The goods on the small train will randomly drop some rewards (later my colleague told me that there are strategies, but I don’t bother to think about it)

3. Click on the building to harvest gold coins, and randomly drop rewards (red envelopes, blue star coins, contribution value)

Based on the above three points, you can Design the framework.

1. First, there must be a while loop running all the time, so that it can automatically collect goods and collect gold coins. Here the while loop is set to run 4s in a loop

2. Then there are several modules in the while loop: collecting images, identifying targets, and moving goods (collecting gold coins).

Collect images

Here we use the Blue Stack simulator to run the game “Dream of Home Country”, because there is no need for a real machine, and it also Support ADB, so it will be very convenient.

At first I thought about opening the Android emulator on the desktop, and then using LabVIEW to take a screenshot of the entire desktop, and then cut out the screenshot of the game. In this way, I also tried to achieve it, but the effect is not very good, because the resolution of the game pattern part that is cut out is different from the resolution of the Android emulator, which means that the emulator compresses the size of the display.

So I simply used ADB to take a screenshot, after the screenshot was uploaded to the PC, and then analyzed. ADB screenshot instructions refer to this post: https://www.cnblogs.com/guoguojiang/p/10347211.html

1. Use ADB to send screenshot instructions (adb shell screencap -p /sdcard/screen. png)

2. Use the ADB command to upload the screenshot to the PC (adb pull /sdcard/screen.png)

3. Use the API in Vision to open the local png file, and prepare to pass it to the subsequent image processing part. The entire code looks like this:

4. Delete the picture just transferred

Identify the target

In the vision toolkit of LabVIEW, there is an API dedicated to pattern matching, so I chose the one I used before The template matches the relevant API. I don’t go into details about this API, because the vision toolkit also comes with examples, which are easy to use. The simplest use process is to learn the template first, and then identify whether there is template information in the target map.

1. First, cut out the difference Cut out all the characteristic part patterns and save them as a png file. It should be noted here that some features may randomly appear on any of the three compartments. The patterns of different compartment templates are slightly different. You need to take more screenshots of pictures in different compartments.

2. The entire algorithm flow is, Load all the templates first (because the load+learn template is slow, so load it first), and then perform geometric template matching on the collected images. If the template group is found, slide it to the corresponding building. If the template is not found, collect the gold coins in the building.

There are some small ways to optimize the algorithm, such as setting the ROI to a small area around the small train, so that the search range will be greatly reduced. Through this matching algorithm, the coordinates of the goods can be found and passed to the subsequent handling module. In the code in the figure below, there are more than a dozen template images and screenshots of our game; the output is the detected position of the goods and the index of the goods, which is to facilitate finding the corresponding building coordinates (the building coordinates are stored well , Can be considered as a hash-map corresponding to the index one-to-one). Here, it is considered that the template is detected when the score is above 900 (the matching score is between 0-1000, and the higher the match, the higher the match).

Moving goods (collecting gold coins)< /h2>

It is very simple to move the goods. Now that you have the coordinates xy of the goods and the coordinates xy of the corresponding buildings, you can use the ADB command similar to the following.

adb shell input swipe 250 250 300 300

When the goods are not found, you can collect gold coins, or send ADB instructions, but Need to collect gold coins for 9 buildings in a row, you can use an ADB command queue to slip the ninth from the first building, and then go back to the first one.

Result

As a result, you can watch this video. In fact, the ADB commands run very slowly, and they are all larger than the loop rate (4S) I set, but let’s do it first.

https://v.youku.com/v_show/id_XNDQwNDY2Mzg0MA==.html?spm=a2hzp.8244740.0.0

You can get 100 red envelopes in one night, probably beat It’s really unexpected that there is a limit on the supply of small trains. . .

Expand

1. Because there will be 200+ red envelopes after running for two nights, it is too tiring to open red envelopes manually, so I simply made an automatic opening Red envelope program. The idea is very simple, it is to use ADB commands to simulate clicking regularly, and it can also support a variety of red envelopes and open the picture album, which saves a lot of effort.

2. Because screenshots are very troublesome, I also wrote a small tool that can save templates easily.

If there are new requirements in the future, let’s continue with new features. The code is not well written, that is, it can be used. Welcome to communicate.