Augmented Reality (AR)

The application of AR takes place when conducting a separate Esports tournament/event/project both online and offline, with live streaming or recorded audiovisual work. By incorporating the characters from the respective video games, an exciting and captivating visual effect can be achieved. Esports events with live streaming or recorded audiovisual work may contain intellectual property rights objects of third parties, including but not limited to copyrighted material such as music, videos of other tournaments, and team logos. To respect the rights of the owners of such objects, it is necessary to obtain the appropriate permit or license from them.

Augmented reality is a consolidated system of software (for example, Unreal Engine[1], and others) and hardware (cameras, servers, and others) means by which 2D and 3D objects are transferred/appear in reality. That is, the real world is continued by the digital world with the help of extension sets.

For this, when conducting online cybersport events, for example, Unreal Engine[2] may be used, in particular, as a real-time transmission environment that allows to create an overlay on cameras. By determining the coordinates of a real camera controlling it in the real world, and sending these coordinates to the digital world, we get the opportunity to control a virtual (not real) camera, which is an exact duplicate of a real camera, taking into account the distortion of the lens, the shift of the optical axis at the extreme points zoom, etc.

A virtual (not real) camera controls an operator who is in the real world and thus it is possible to subtly connect the two worlds by moving the camera from the real world to the digital (not real) world. And thus, we get the result – a rendered picture in the real world.

The process of creating and implementing AR during live streaming of Esports events can be conditionally divided into the following stages:

A1. development and creation of design, elements of graphic design, author’s 2D and 3D objects;

A2. production activities for bringing 2D and 3D objects from the digital world into the real world in the conditions of live streaming;

A3. post-production activities for the development and creation using computer graphics – titles, graphic elements of layers of augmented reality, animation, special effects, processing, editing, and composition of audiovisual works;

A4. software development, modification, testing, and technical support, to provide augmented reality production in the context of live streaming. Ensuring the performance of optical tracking systems by creating distortion maps and writing control scripts.

The development of augmented reality animation belongs to the creation of objects of intellectual property rights (characters, pictures, logo etc.) and includes:

• Creation of storyboards (storyboards) for animation;
• Creating characters;
• Creation of animation elements.

Creation of storyboards

Creating animation begins with the process of creating storyboards. This is a prototype of future animation. A storyboard is a sequence of drawings, each of which displays a corresponding scenario frame from the video, together they create a preliminary visualization of the entire video. Storyboards for animations can have multiple layers. Characters and backgrounds are separated into individual elements that are moved to layers in the software’s editing tools. Next, each element can be animated. Drawings resulting from this process can range from sketches of highly detailed shapes and objects to simple shapes that help demonstrate an important position or action.

Creating characters for animation

The development of 3D models of multi-visual animated characters combined with augmented reality and visual modeling technology provides the design of 3D models of multi-visual animated characters. The process of modeling each character may include the following stages:

• Concept and design;
• Polygonal grid modeling and sculpting;
• Retopology;
• Sweeping;
• Texturing.

A character is a fictional creature with certain defined characteristics of behavior and character and a corresponding appearance. The first step in modeling a character is to approve references (examples that you can refer to, photos, or pictures that reflect the idea). In the process of developing a character concept, a unique image, style, and character transfer with a high level of detail is created. After that, 3D modeling of the character begins – the process of creating a digital, three-dimensional model of the object using specialized software.

Creating animation elements includes several stages. The first step is to conceptualize the idea and determine the interaction with the animated element in the augmented reality environment. The next step is the development and design of three-dimensional objects, which combines many technical processes, such as texturing, working with forms, rendering, etc. The last step is the preparation of 3D models for animation according to the approved scenarios.

[1] ‘Augmented Reality Solutions’ (Pixotope) <https://www.pixotope.com/solutions/augmented-reality>accessed 05 March 2024

[2] ‘Augmented Reality Solutions’ (Pixotope) <https://www.pixotope.com/solutions/augmented-reality>accessed 05 March 2024