- Dec 25, 2022 Update structure, fix typo
- Jul 09, 2023 Separate JA and EN articles
Yunit5.3 Offensive robot of team Mavericks (10th Place in RCJ Japan Open)
Background and Introduction
This article introduces the method of building a hyperbolic-mirror-based omnidirectional camera with students' own hands. The omnidirectional camera has a strong advantage in its wide field of view that allows robots to keep detecting the ball in whole gameplay, and the calculation of determining ball position is much easier compared to the way utilizing multiple narrow-angle cameras. The article explains how can you build a 3D curved shaped mirror using the method "heat forming" as well as the video example of how the camera view will look like.
One important background of building omnidirectional cameras is a restriction in using COTs cameras. Building the sensor with a student's own effort is required in RoboCupJunior soccer's rule which was first published in 2017 and continues to now (as of 2023). Under the RoboCupJunior soccer rule, you can NOT use COTs omnidirectional cameras which are specifically defined as cameras with more than 140° horizontally and 80° vertically field of view. You can use omnidirectional cameras as long as the camera device was designed and made by the students' own hands-on work. The motivation for limiting COTs omnidirectional cameras was originally preventing the money game. However, the price of wide-angle COTs camera is not considerably expensive as the years progresses, so now it is a bit skeptical that this limitation will prevent the money game. Yet, it still is effective to encourage students to think and learn about the method of building specific hardware.
All commercial omnidirectional lenses/cameras are not permitted.
Only omnidirectional lenses/cameras made by students are permitted, meaning that their construction needs to be primarily and substantially the original work of a team.
The robot at the top, Yunit5 was built for the 2017 Japan domestic competition. The passive ball rule was not adopted in Japan's competition as of 2017, but I tried an omnidirectional camera in a trial. After the domestic tournament, our team shared the method of building omnidirectional cameras to team INPUT, the winner of the 2017 Japan tournament. They won the same year's world championship with an omnidirectional camera built in the same way as introduced in this article.
Designing Mirror Mold
Figure 1 shows the mechanical design of an omnidirectional camera. The camera has a mirror at the top of the cylinder, and the camera will be installed coaxial to the cylinder and face to mirror upward. In this design, we used a transparent acrylic cylinder, but you may also use non-transparent materials as long as the size of the frame blocking the camera view can be minimized.
Fig.1 Self-made 360 degree camera 3D CAD design
The way of designing hyperbolic mirrors is not the only one. If you are good at math, you may be able to find an equation and design the curve parametrically. In my case, I was not good at math and finally gave up composing the equation, and finally made up my mind to draw the reflection surface on the 2D cad one by one, as shown in Figure 2. If you are seeking a way to calculate the curve using an equation, this document will help to understand.
Fig.2 Drawing hyperbolic curve utilizing 2D-CAD
Figure 3 shows the constraint about the AOI and AOR on AutoCAD calculation. If you are using 2D/3D cad, you can draw the curve by repeating and extending the curve surface in sufficiently short length with constraints about AOI and AOR. If you know how to use AutoCAD or EXCEL, you can use macros and this process won't take a long time.
Fig.3 The relationship between AOI(*) and AOR(**) on the hyperbola mirror
- (*)Angle of incident
- (**)Angle of reflection
If you are drawing the hyperbolic curve in a proper way, you may find an exciting behavior about the light axis. The length of the light axis will result in the same length in the all way from the focus point to the end of the view surface after the reflection.
Fig.4 The mathematical property of length of the optical axis
Typical parameters of a hyperbolic-mirror-based omnidirectional camera will be the followings:
- Omnidirectional camera field of view
- Inner/Outer diameter of hyperbolic mirror
- Either of the following two parameters
- Camera field of view
- Distance between the focus point of lens and pole of hyperbolic mirror
Fig.5 The parameter for calculate Hyperbola-mirror
Building a Mold and Forming Mirror
We formed the 3D-shaped mirror using "heat forming", which is typically used to form simple 3D shapes from plastic sheeting. It is generally known as a way to form thin plastic sheeting to a specific shape by pressing the sheet onto the mold after applying heat and making it elastic. If the shape is more complex, you can also use the "vacuum forming" method which also combines the heated plastic sheeting, mold, and equipment to apply suction to the sheeting. However, for a simpler shape such as a hyperbolic mirror mold, you may not need to use the vacuum.
Figure 6 shows the 3D data of the mold we used to form the mirror. If you already have the cross-section shape of the hyperbolic curve, all you need to do is just rotate that sketch on the 3D-CAD. Also, this is better to have the obvious inner and outer diameter boundary on the mold so you can find where to apply scissors when you cut.
Fig.6 The hyperbola curve surface 3D CAD Model
Figure 7 shows the mold we built. We used a 3D printer to build the mold and apply the plastic paste on the contact surface, and file it so the surface becomes smooth. We have never tested it but it may not be a good idea to use the raw 3D printed mold directly to the heat-forming mold because the surface roughness of 3D printed parts is generally very low and it might affect the image quality of a camera, and the 3D printed parts have no resistance against the heat.
When you apply heat to the plastic sheeting, you can use an electric heater or hot air. For safety reasons, it is not a good idea to use a gas stove. The thin plastic sheeting burns up much easier than you imagine.
Fig.7 The mold for plastic heat forming
Detailed specifications of the plastic mirror sheeting we used in this article are specified below:
- Thickness: 0.5mm
- Material: Transparent Vinyl with Mirrored Plating
- Cutting size: 150 x 150 mm (outer diam of mold = 50 mm)
Some have asked us "How do I get a plastic mirror sheet which is appropriate for heat forming?", however, procurement will be each different depending on the country or region you are belonging to. Moreover, I have yet to learn about a Japanese shop/supplier that is selling similar mirror sheeting globally.
If we define the requirements of the plastic sheeting that can be used for heat forming, those below will be important things to check.
- It becomes soften when you heat it up -- Vinyl, Polycarbonate, Pet, etc
- Thickness around 0.5~1.0 mm -- Thinner or thicker is not suit for heat forming
- Mirror keeps fine even you heat it up -- some plastic mirror become cloudy when heat is applied or mirror plating get clacked when it is expanded
Fig.8 A vinyl chloride mirror sheet
Fixing the mirror sheet to the fixture is a very important process to form a mirror with good quality. Note that the size of the plastic sheeting and fixing the outer edge of the sheeting are crucial for the quality. Through our experience, we would recommend having at least twice the outer diameter of the mold, triple is even better. We also made an MDF sheeting with a circled window to fix the outer edge of the mirror sheet, and that helps to generate equal pressure between mold and plastic sheeting.
One more question will be which of the top or the bottom side we can push into the mold. We have no concrete answer to this question because it totally depends on what type of materials and mirror plating you are about to use. You may need to practice to get a good result anyways, so let's just try it and determine the best way to form.
Fig.9 Cutting the fixture with circular window
Fig.10 Taping mirror sheet to the fixture
Fig.11 Mirror sheeting with fixture, this whole set will be press into the mold after heat is applied
You will see the result of the mirror once it cooled, very shortly. Some may see the problem, such as a clack of mirror plating or smoke on the transparent plastic, which usually occur when the material or plating was not sufficient for the heat-based forming. In case of insufficient material or plating selections, it is better to try different materials rather than changing the heat conditions. The mirror-plating plastic sheet is not designed to form by heating up, so something unexpected always happens. There is no other way to determine sufficiency except for trial and error.
Movie1 Plastic sheet heat press forming
Fig.12 Detaching from the fixture and cut
Fig.13 Detached mirror and put it on the mold
Integrate to Camera Module
We have used Pixy CMUcam5 as a color detector module. This device provides an integrated image sensor as well as color detector software and standard communication interfaces such as UART, I2C, and SPI. In recent years, many participants utilize single board computers such as Raspberry Pi or Jetson TX to obtain higher image processing capabilities, which I think is better if you are wanting to expand the limit in resolution or frame rate. Through our experience of using Pixy, we sometimes had difficulties in detecting the ball in longer distances (particularly more than 2m) due to the resolution limit. Particularly, if the robot needs to perform in the big field of the world championship too, low resolution and limited color detector algorithms will be a big problem.
Movie 2 shows how Pixy CMUcam5 detects the color signature and returns the coordinate of the color signature to the microcontroller. The module can detect multiple color signatures and return each position in the image respectively.
Mov.2 Pixy test 1 (Read with Arduino), from kemarin-tech.blog.jp
Figure 15 and 16 shows the mechanical frame of the omnidirectional camera module. We asked Japanese supplier acry-ya.com for cutting and selling the transparent cylinder. They provide various inner and outer diameter sizes as well as high-precision cutting services.
When we use the Pixy camera module, we need to define the color code every time before the game starts. That is because the tolerance of color is very narrow and even small difference in light conditions in each different soccer field affects the perception very much. In order to make the calibration process easier, we secured the USB cable path and allow the user to connect to the camera even in the assembled configuration. This process may easier if you are equipping the SBC with the operating system because this process can be done remotely via a local network.
See How It Works
Movie 3 shows how the sensor works as an omnidirectional camera in an assembled configuration. You already find a design problem with Yunit5 which is the shape of chassis blocking the view of the camera when the ball is in closer distance. This is because 2017 Japan domestic competition was held with an IR ball and robots did not need to detect the color ball. The omnidirectional camera of Yunit5 was only designed and equipped for goal detection purposes.
If the robot needs to detect the ball and keep track of it, the view of omnidirectional camera must be secure well enough. A good example in terms of chassis design will be a RoboCup Middle Size League robot. They have a cone-shaped design and there is no chassis blocking the camera view, and that I personally believe that design should be adopted in the Junior Soccer Open League as well.
Mov.3 Self-made 360° camera demonstration
Figure 18 shows the maximum distance that the camera can detect the ball. This limitation is mostly from its low image resolution, but also from the color detection algorithm. In the small field, this limitation is acceptable, but it may not be in the big field of the world tournament.
Fig.18 Ball detectable max limit distance
Figure 19 and 20 shows the minimum distance that robot can detect the ball. It is possible to control robots even if the robot has this issue, however, it may make the software more complicated. As an overall strategy for building robots, it would be way better to design hardware not to block the view of the camera.
When we tried to build the control software of this robot, we struggle to detect the ball continuously due to this problem. The software of Yunit5 is state-machine-based algorithm, and continuous ball detection is a fundamental condition of state distribution.
Fig.19 Ball detectable min limit distance (in front of Robot)
Fig.20 Ball detectable min limit distance (behind of Robot)
In the End
It has been a very long time since we published this article in 2017, and so far, many teams visited this article and built their sensor in the same way or a bit modified ways. Contribution to the communities is always fun and I also could learn a lot through these activities. At the same time, I would like everyone reading this article to be involved in this loop of sharing information and raising common technical knowledge. Thank you very much.
コメント