Shark 3D Live – Virtual Production Solution Spinor

Create and run interactive virtual 3D worlds

• Joining forces: Movie, game, broadcasting and virtual reality production – Shark 3D Live combines possibilities and features of professional movie production software, high-end game technologies and broadcasting production solutions. This unique combination opens new levels of creativity and productivity in each of these areas.
• Flexibility and agility without dead ends – Shark 3D Live is designed for agility and flexibility from the bottom up. It includes a real-time 3D authoring editor and a cross-platform 3D engine. It can also be used as middleware and software platform.
• Proven reliability – Shark 3D Live is used successfully in a large variety of projects on various software and hardware platforms in areas like video games, films and series, broadcasting and interactive applications.

Authoring editor

• Tree-based scene setup
• Hierarchical organization of nodes
• Lifabs: Smart live prefabs and templates for reusing entities and functionality
• Unlimited nesting of Lifabs
• Live editing of Lifabs
• Customization of Lifabs via slots
• Support for defining custom level managements
• Support for defining new 3D states and custom view ports
• Inspection of run-time instances of Lifabs
• Python shell for controlling the editor
• Python shell for interactively manipulating runtime components
• Python scripts can control the editor in window-less command line mode

Recording, track editing, and rendering

• Feature-based recording and replaying actions in the 3D world
• Per channel control over what is recorded, what is replayed, and what is calculated or simulated live at the same time
• Recording and replaying sub-channels, for example replaying a raw camera movement combined with real-time generation of steadicam movements with different settings
• Recording and replaying resulting channels, for example total camera movements including steadycam movements
• Physics-based replaying, which can react to and influence other entities or the environment
• Adaption of run-time simulation to edited recordings, for example adapt walk animations live to an edited character path
• Slow motion and time lapse simulation, recording and replaying
• Hierarchical structure of tracks and shots
• Cutting, blending, timing, and scaling of tracks
• Non-destructive track editing
• Non-linear track editing
• Track editing on any hierarchy level like the whole scene, groups of characters and animated objects, individual characters and animated objects, groups of animation channels, or individual animation channels
• Block-based interleaving sparse track storage optimized for real-time reading, writing, seeking and live non-destructive recombining high numbers of channels containing high numbers of key frames
• Large recordings don’t need to fit into memory but are automatically streamed to and from disk when recording, playing or editing

Output

• Packaging as stand-alone application (e.g. game)
• Live low latency H.264 (raw or fragmented mp4) streaming via TCP and WebSocket
• Render to a sequence of high resolution image files
• Render to high resolution video file, supporting different codecs
• Semi-real-time GPU-based rendering with enhanced quality settings
• Virtual reality head-mounted display headsets, including six degrees of freedom rotational and positional tracking
• 360 degree panorama rendering
• Stereo 3D rendering
• Multi-channel rendering

Input

• Gamepads, joysticks, and other controllers
• Video-in
• Camera tracking
• Camera lens distortion correction and calibration
• Camera lens focal length measurment
• Control input via network
• Synchronization with video-in, genlock and tracking, including individually configurable delays with buffering
• APIs for integrating and synchronizing any input source

Cloud support

• Server-side processing (including rendering), streaming input from and output to thin clients
• Multiple isolated or collaborative users per server-side engine instance
• Extensive resource sharing between multiple users, reducing clould infrastructure costs to a mere fraction

Renderer

• All features are completely real-time (no precalculation waiting times after scene changes)
• All kinds of objects are handled in the same way (no need for differentiating between static and dynamic objects)
• Physically based rendering including energy conservation, albedo, reflectivity, glossiness and Fresnel
• Reflectivity workflow, metalness workflow and custom workflows
• Light-transport-based rendering of soft lighting
• Omnidirectional and spot lights
• Extended light sources casting contact hardening soft shadows
• Reflection and refraction of flat and curved surfaces
• Global illumination effects like interreflection, color bleeding, and ambient occlusion
• Lighting of complex materials like strands of hair, skin, fabric or brushed metal
• Efficient detailed shapes like fur or hair
• High dynamic range effects
• Bokeh depth of field, with efficient support of large apertures, and without halo or bleeding
• Bloom
• Particles
• Fog
• Node-based shader editor
• Physically based rendering can be accessed via multiple nodes which can be used and combined flexibly in different ways for special needs
• TrueType Unicode text rendering
• Extruded 3D text, based on TrueType Unicode, including bevels and mapping coordinates
• Shader animation
• Chroma keyer
• Video textures (video plates)
• All per-vertex and per-pixel rendering operations are running completely on the GPU

Animation system

• Node-based animation system
• Hierarchical animation fading and blending
• Import of path animations
• Path animation editor integrated in authoring editor
• Import of skeleton animations
• Flexible skinning of multiple objects by multiple skeletons
• Any number of attachment points of skeletons
• Retargeting animations to different skeletons
• Walk animation adaption in all directions
• Mesh animations (morphing)
• Compact storage and efficient execution of sparsely and intermittently animated meshes
• Blend shapes (morph targets)
• Unlimited number of simultaneous active blend shapes
• Compact storage and efficient execution of overlaying many sparse blend shapes
• All per-vertex and per-pixel animation operations are running completely on the GPU

Audio

• 3D sound
• Physics collision and friction sound effects

Physics

• Collision geometry built of collision meshes and primitive geometries
• Various types of constraints like for example ball-and-socket, hinge and hinge-two, or for keeping a character vertical
• Motors and character movers
• Player control elements
• Physics driven skinning
• Ragdoll
• Networking integrated with physics

Networking

• Integrated networking
• Server-side versus client-side execution
• Lag suppression for player movements
• Remote command execution
• Optimizations for avoiding network traffic when loading levels
• Adaptive network traffic compression

Logic

• Command handling and dispatching
• Generic triggers and sensors
• Sandbox based scripting language “Perch” designed for easily accessing engine components and integrated protection against memory leaks and crashes
• Python

Component architecture

• Component architecture based on composition, encapsulation, name spaces, dynamic command flows, and scripting bindings
• Thread-safe architecture from bottom up
• Thread-safe dependency graphs supporting delayed evaluation
• Support for streaming, background loading, and world part states onto disk in separate thread
• Generic dynamic method invocation APIs for C++ components, which can be accessed for example via Python
• Universal and flexible serialization mechanism
• Support for adding serialization for existing external C++ objects which are not prepared for serialization
• Resource manager abstraction supporting individual files, pack files, and TCP
• Universal efficient hierarchical snake data file format, with equivalent text and binary representations, optimized for efficiency and simplicity
• Unicode based (UTF-8) from the bottom up
• SAP (Simple Access Protocol) as generic and easy-to-parse remote method invocation protocol, supporting also callback objects

Target platforms

• PC (Windows, Linux)
• Mobile devices
• Consoles
• Browser plugin

Documentation

• Introduction articles
• Tutorials
• Reference sections
• Keyword search
• Context sensitive integration with the authoring editor
• As HTML (~25k pages) or pdf (~10k pages)

Reliability

• Based on very high software design and coding standards
• Focus on clean and robust software architectures
• Designed from the bottom up for stablility without compromising performance

Software platform

• Designed, from the bottom up, as an universal software platform for creating new modules, features, customizations, extensions and live connections to other software
• Designed for flexibility from the bottom up
• Modular system for a large variety of applications in different markets
• High-level Python APIs
• Low-level C++ APIs
• Custom modules can access and use the same APIs as all our standard Shark 3D component implementations (we at Spinor don’t use hidden internal APIs)
• Custom modules can access standard Shark 3D components and vice versa
• Live-Live Editing (LLE) to update changes in the Python editor code (e.g. custom editor nodes) live into the running editor for fast customization testing turn-around times of the editor
• Generic and flexible editor development toolkit for creating own customized editors
• APIs for defining new recording and replay channels
• Flexibility of defining how replaying recordings interact with the simulation of the 3D world
• Generic synchronization architecture
• Custom rendering components can access the whole scene database and can interact with all rendering steps
• Custom animation components can access, modify or build new animation, skeleton and bone transformation dependency graphs
• APIs for live input of animation data from other software, for example live motion capture data
• Custom components can implement individual custom networking synchronization algorithms and protocols
• Generic APIs for custom script language bindings
• Generic APIs to asset and project management software
• TCP protocol for live updating external run-time resources

Software architecture

• Clean modular multi-layer architecture
• Parallelized Dependency Graph (PDG) processing the virtual 3D world live efficiently
• Rapid Construction Kit (RCK) manipulating and controlling the Parallelized Dependency Graph
• Live incremental translation of nodes and Lifabs into Rapid Construction Kit components
• Re-entrant Modular Shader System (RMSS)

Renderer hardware

• Rack mountable in 4U chassis
• Processor: Intel Core i7 4790K 1150, 8 MB Cache, 4,0 GHz
• Main memory: 32 GB DDR3 memory
• Graphics card:
  GeForce
  PCIe ASUS STRIX-GTX 980 TI
  GM200-310-A1 “Maxwell”
  Processor rate 1216MHz
  Memory 6GB GDDR5, 1800MHz
  384bit, 346GB/s
  Shader units /TMUs/ROPs: 2816/176/96
  Computing power 6849 GFLOPS (Single), 214 GFLOPS (Double)
• Video in/out:
  DeckLink 4K Extreme 12G
  Capture and playback 12G-SDI in SD, HD, Ultra HD and 4K DCI up to 4096 x 2160 60p in 10-bit YUV or 12-bit RGB
• Storage: 500 GB SSD and 4 TB HDD

Video plates

Multiple video plates up to 4096×4096, GPU-accelerated decoding

Specs

Support & Maintenance

Shark 3D Live Virtual Production Solution includes 5 days on-site training.