Rendering huge biological scenes with atomistic detail presents a significant challenge in molecular visualization due to the memory limitations inherent in traditional rendering approaches. In this paper, we propose a novel method for the interactive rendering of massive molecular scenes based on hardware-accelerated ray tracing. Our approach circumvents GPU memory constraints by introducing virtual instantiation of full-detail scene elements. Using instancing significantly reduces memory consumption while preserving the full atomistic detail of scenes comprising trillions of atoms, with interactive rendering performance and completely free user exploration. We utilize coarse meshes as proxy geometries to approximate the overall shape of biological compartments, and access all atomistic detail dynamically during ray tracing. We do this via a novel adaptive technique utilizing a volumetric shell layer of prisms extruded around proxy geometry triangles, and a virtual volume grid for the interior of each compartment. Our algorithm scales to enormous molecular scenes with minimal memory consumption and the potential to accommodate even larger scenes. Our method also supports advanced effects such as clipping planes and animations. We demonstrate the efficiency and scalability of our approach by rendering tens of instances of Red Blood Cell and SARS-CoV-2 models theoretically containing more than 20 trillion atoms.

We introduce DiffFit, a differentiable algorithm for fitting protein atomistic structures into an experimental reconstructed Cryo-Electron Microscopy (cryo-EM) volume map. In structural biology, this process is necessary to semi-automatically composite large mesoscale models of complex protein assemblies and complete cellular structures that are based on measured cryo-EM data. The current approaches require manual fitting in three dimensions to start, resulting in approximately aligned structures followed by an automated fine-tuning of the alignment. The DiffFit approach enables domain scientists to fit new structures automatically and visualize the results for inspection and interactive revision. The fitting begins with differentiable three-dimensional (3D) rigid transformations of the protein atom coordinates followed by sampling the density values at the atom coordinates from the target cryo-EM volume. To ensure a meaningful correlation between the sampled densities and the protein structure, we proposed a novel loss function based on a multi-resolution volume-array approach and the exploitation of the negative space. This loss function serves as a critical metric for assessing the fitting quality, ensuring the fitting accuracy and an improved visualization of the results. We assessed the placement quality of DiffFit with several large, realistic datasets and found it to be superior to that of previous methods. We further evaluated our method in two use cases: automating the integration of known composite structures into larger protein complexes and facilitating the fitting of predicted protein domains into volume densities to aid researchers in identifying unknown proteins. We implemented our algorithm as an open-source plugin (github.com/nanovis/DiffFit) in ChimeraX, a leading visualization software in the field.

We present a novel method for the interactive construction and rendering of extremely large molecular scenes, capable of representing multiple biological cells in atomistic detail. Our method is tailored for scenes, which are procedurally constructed, based on a given set of building rules. Rendering of large scenes normally requires the entire scene available in-core, or alternatively, it requires out-of-core management to load data into the memory hierarchy as a part of the rendering loop. Instead of out-of-core memory management, we propose to procedurally generate the scene on-demand on the fly. The key idea is a positional- and view-dependent procedural scene-construction strategy, where only a fraction of the atomistic scene around the camera is available in the GPU memory at any given time. The atomistic detail is populated into a uniform-space partitioning using a grid that covers the entire scene. Most of the grid cells are not filled with geometry, only those are populated that are potentially seen by the camera. The atomistic detail is populated in a compute shader and its representation is connected with acceleration data structures for hardware ray-tracing of modern GPUs. Objects which are far away, where atomistic detail is not perceivable from a given viewpoint, are represented by a triangle mesh mapped with a seamless texture, generated from the rendering of geometry from atomistic detail. The algorithm consists of two pipelines, the construction-compute pipeline, and the rendering pipeline, which work together to render molecular scenes at an atomistic resolution far beyond the limit of the GPU memory containing trillions of atoms. We demonstrate our technique on multiple models of SARS-CoV-2 and the red blood cell.

We present a novel smart visibility system for visualizing crowded volumetric data containing many object instances. The presented approach allows users to form groups of objects through membership predicates and to individually control the visibility of the instances in each group. Unlike previous smart visibility approaches, our approach controls the visibility on a per-instance basis and decides which instances are displayed or hidden based on the membership predicates and the current view. Thus, cluttered and dense volumes that are notoriously difficult to explore effectively are automatically sparsified so that the essential information is extracted and presented to the user. The proposed system is generic and can be easily integrated into existing volume rendering applications and applied to many different domains. We demonstrate the use of the volume conductor for visualizing fiber-reinforced polymers and intracellular organelle structures.

We propose a new microscopy simulation system that can depict atomistic models in a micrograph visual style, similar to physical electron microscopy imaging results. This system is scalable, able to represent the simulation of electron microscopy of twenty complex viral particles, and synthesizes the image faster than previous approaches. Additionally, the simulator is differentiable in the deterministic and stochastic stages that form signal and noise representations in the micrograph. This allows for solving inverse problems by means of optimization and thus allows for the generation of microscopy simulations using parameter settings estimated from real data. We demonstrate this capability through two applications: (1) estimating the parameters of the modulation transfer function defining the detector properties of the simulated and real micrographs and (2) denoising the real data based on parameters optimized from the simulated examples using gradient descent. While current simulators do not support any parameter estimation due to their forward-only design, we show that the results obtained using estimated parameters are very similar to real micrographs. Additionally, we evaluate the denoising capabilities of our approach and show that the results are competitive with state-of-the-art methods. Denoised micrographs exhibit less noise in the tomographic reconstruction of tilt-series, facilitating visualization of microscopy tomography using direct volume rendering by reducing the visual dominance of noise.

Electronic immunosensors are indispensable tools for diagnostics, particularly in scenarios demanding immediate results. Conventionally, these sensors rely on the chemical immobilization of antibodies onto electrodes. However, globular proteins tend to adsorb and unfold on these surfaces. Therefore, self-assembled monolayers (SAMs) of thiolated alkyl molecules are commonly used for indirect gold–antibody coupling. Here, a limitation associated with SAMs is revealed, wherein they curtail the longevity of protein sensors, particularly when integrated into the state-of-the-art transducer of organic bioelectronics—the organic electrochemical transistor. The SpyDirect method is introduced, generating an ultrahigh-density array of oriented nanobody receptors stably linked to the gold electrode without any SAMs. It is accomplished by directly coupling cysteine-terminated and orientation-optimized spyTag peptides, onto which nanobody-spyCatcher fusion proteins are autocatalytically attached, yielding a dense and uniform biorecognition layer. The structure-guided design optimizes the conformation and packing of flexibly tethered nanobodies. This biolayer enhances shelf-life and reduces background noise in various complex media. SpyDirect functionalization is faster and easier than SAM-based methods and does not necessitate organic solvents, rendering the sensors eco-friendly, accessible, and amenable to scalability. SpyDirect represents a broadly applicable biofunctionalization method for enhancing the cost-effectiveness, sustainability, and longevity of electronic biosensors, all without compromising sensitivity.

Cryo-electron tomography (cryo-ET) is a new 3D imaging technique with unprecedented potential for resolving submicron structural details. Existing volume visualization methods, however, are not able to reveal details of interest due to low signal-to-noise ratio. In order to design more powerful transfer functions, we propose leveraging soft segmentation as an explicit component of visualization for noisy volumes. Our technical realization is based on semi-supervised learning, where we combine the advantages of two segmentation algorithms. First, the weak segmentation algorithm provides good results for propagating sparse user-provided labels to other voxels in the same volume and is used to generate dense pseudo-labels. Second, the powerful deep-learning-based segmentation algorithm learns from these pseudo-labels to generalize the segmentation to other unseen volumes, a task that the weak segmentation algorithm fails at completely. The proposed volume visualization uses deep-learning-based segmentation as a component for segmentation-aware transfer function design. Appropriate ramp parameters can be suggested automatically through frequency distribution analysis. Furthermore, our visualization uses gradient-free ambient occlusion shading to further suppress the visual presence of noise, and to give structural detail the desired prominence. The cryo-ET data studied in our technical experiments are based on the highest-quality tilted series of intact SARS-CoV-2 virions. Our technique shows the high impact in target sciences for visual data analysis of very noisy volumes that cannot be visualized with existing techniques.

We present a novel framework for 3D tomographic reconstruction and visualization of tomograms from noisy electron microscopy tilt-series. Our technique takes as an input aligned tilt-series from cryogenic electron microscopy and creates denoised 3D tomograms using a proximal jointly-optimized approach that iteratively performs reconstruction and denoising, relieving the users of the need to select appropriate denoising algorithms in the pre-reconstruction or post-reconstruction steps. The whole process is accelerated by exploiting parallelism on modern GPUs, and the results can be visualized immediately after the reconstruction using volume rendering tools incorporated in the framework. We show that our technique can be used with multiple combinations of reconstruction algorithms and regularizers, thanks to the flexibility provided by proximal algorithms. Additionally, the reconstruction framework is open-source and can be easily extended with additional reconstruction and denoising methods. Furthermore, our approach enables visualization of reconstruction error throughout the iterative process within the reconstructed tomogram and on projection planes of the input tilt-series. We evaluate our approach in comparison with state-of-the-art approaches and additionally show how our error visualization can be used for reconstruction evaluation.

Immersive virtual reality environments are gaining popularity for studying and exploring crowded three-dimensional structures. When reaching very high structural densities, the natural depiction of the scene produces impenetrable clutter and requires visibility and occlusion management strategies for exploration and orientation. Strategies developed to address the crowdedness in desktop applications, however, inhibit the feeling of immersion. They result in nonimmersive, desktop-style outside-in viewing in virtual reality. This article proposes Nanotilus-a new visibility and guidance approach for very dense environments that generates an endoscopic inside-out experience instead of outside-in viewing, preserving the immersive aspect of virtual reality. The approach consists of two novel, tightly coupled mechanisms that control scene sparsification simultaneously with camera path planning. The sparsification strategy is localized around the camera and is realized as a multi-scale, multi-shell, variety-preserving technique. When Nanotilus dives into the structures to capture internal details residing on multiple scales, it guides the camera using depth-based path planning. In addition to sparsification and path planning, we complete the tour generation with an animation controller, textual annotation, and text-to-visualization conversion. We demonstrate the generated guided tours on mesoscopic biological models - SARS-CoV-2 and HIV. We evaluate the Nanotilus experience with a baseline outside-in sparsification and navigational technique in a formal user study with 29 participants. While users can maintain a better overview using the outside-in sparsification, the study confirms our hypothesis that Nanotilus leads to stronger engagement and immersion.

We present a new technique for the rapid modeling and construction of scientifically accurate mesoscale biological models. The resulting 3D models are based on a few 2D microscopy scans and the latest knowledge available about the biological entity, represented as a set of geometric relationships. Our new visual-programming technique is based on statistical and rule-based modeling approaches that are rapid to author, fast to construct, and easy to revise. From a few 2D microscopy scans, we determine the statistical properties of various structural aspects, such as the outer membrane shape, the spatial properties, and the distribution characteristics of the macromolecular elements on the membrane. This information is utilized in the construction of the 3D model. Once all the imaging evidence is incorporated into the model, additional information can be incorporated by interactively defining the rules that spatially characterize the rest of the biological entity, such as mutual interactions among macromolecules, and their distances and orientations relative to other structures. These rules are defined through an intuitive 3D interactive visualization as a visual-programming feedback loop. We demonstrate the applicability of our approach on a use case of the modeling procedure of the SARS-CoV-2 virion ultrastructure. This atomistic model, which we present here, can steer biological research to new promising directions in our efforts to fight the spread of the virus.

Studying the transport paths of ligands, solvents, or ions in transmembrane proteins and proteins with buried binding sites is fundamental to the understanding of their biological function. A detailed analysis of the structural features influencing the transport paths is also important for engineering proteins for biomedical and biotechnological applications. CAVER Analyst 2.0 is a software tool for quantitative analysis and real-time visualization of tunnels and channels in static and dynamic structures. This version provides the users with many new functions, including advanced techniques for intuitive visual inspection of the spatiotemporal behavior of tunnels and channels. Novel integrated algorithms allow an efficient analysis and data reduction in large protein structures and molecular dynamic simulations.

The transport of ligands, ions or solvent molecules into proteins with buried binding sites or through the membrane is enabled by protein tunnels and channels. CAVER Analyst is a software tool for calculation, analysis and real-time visualization of access tunnels and channels in static and dynamic protein structures. It provides an intuitive graphic user interface for setting up the calculation and interactive exploration of identified tunnels/channels and their characteristics.

Proteins are essential for functioning of all living organisms and studying their inner structure and functions has been of a high importance. Many studies concentrated on detection of various inner structures inside macromolecules (e.g. tunnels, channels, pores) which play an essential role in the functioning of a large number of proteins. Here we present a novel approach to a detection of intramolecular tunnels. These pathways may facilitate the transport of reaction intermediates among buried active sites. The results obtained by the proposed algorithm were compared to intramolecular tunnels whose presence in given structures is already known. The algorithm is able to also identify other inner structures, such as channels or pores.

In this paper, we present a novel algorithm for the detection of pathways connecting two or more specific user defined binding sites, which are deeply buried in a protein macromolecule. These pathways can play an important role in the protein reactivity and overall behavior. However, our new algorithm can be generalized and used for computation of pathways inside an arbitrary set of spheres in three-dimensional space, leading through an ordered set of user-defined sites. Our approach is based on the localized Voronoi diagram approach and the Delaunay triangulation. The greatest benefit of our approach is its independence on the size of the input data set. This is achieved by using only a subset of all atoms in the macromolecule in each phase. This substantially reduces the size of the processed space. The method can also be utilized for determination whether pathways wide and straight enough exist among determined binding sites. This information then serves as the guideline for assessing the migration of products of chemical reaction between these binding sites.

Analysis and visualization of molecules and their structural features help biochemists and biologists to better understand protein behavior. Studying these structures in molecular dynamics simulations enhances this understanding. In this paper we introduce three approaches for animating specific inner pathways composed of an empty space between atoms, called tunnels. These tunnels facilitate the transport of small molecules, water solvent and ions in many proteins. They help researchers understand the structure-function relationships of proteins and the knowledge of tunnel properties improves the design of new inhibitors. Our methods are derived from selected tunnel representations when each stresses some of the important tunnel properties — width, shape, mapping of physico-chemical properties, etc. Our methods provide smooth animation of the movement of tunnels as they change their length and shape throughout the simulation.

Long-term research in the area of protein analysis proved the importance of an empty space situated inside these macromolecular structures. This empty space influences the protein function, characteristics or reactivity. Many algorithms enabling computation of these empty spaces (or voids) have been published and their results were evaluated by protein engineers to confirm their chemical relevance. However, not all detected voids inside protein are of the same importance. Thus, the examination and assessment of all voids must follow to reveal the important ones. In this phase the visual representation of voids is very valuable and substantially decreases the time spent in this evaluation phase. In this paper we present an extension of the algorithm for the visualization and further evaluation of protein voids in real-time. The userdriven approach enables to compute and display empty space that satisfies the input parameters instantly. The values of these parameters can be changed by the user anytime and the changes are immediately displayed and prepared for further exploration. Our improvements involve an exclusion of selected atom or group of atoms (ligands, ions) from the computation, which can change the size and shape of the detected void. Another improvement is related to the detection of the binding site which is usually located in one of the largest voids. So the algorithm suggests and visually separates (by different coloring) the largest void of given area. Several improvements were also made in the field of real-time exploration – currently the interaction on large structures is fluent. In consequence, the current version of the algorithm provides the biochemists with very adjustable and precise algorithm for detection of inner voids in a user-defined region of protein structures.

Tunnels and channels facilitate the transport of small molecules, ions and water solvent in a large variety of proteins. Characteristics of individual transport pathways, including their geometry, physico-chemical properties and dynamics are instrumental for understanding of structure-function relationships of these proteins, for the design of new inhibitors and construction of improved biocatalysts. CAVER is a software tool widely used for the identification and characterization of transport pathways in static macromolecular structures. Herein we present a new version of CAVER enabling automatic analysis of tunnels and channels in large ensembles of protein conformations. CAVER 3.0 implements new algorithms for the calculation and clustering of pathways. A trajectory from a molecular dynamics simulation serves as the typical input, while detailed characteristics and summary statistics of the time evolution of individual pathways are provided in the outputs. To illustrate the capabilities of CAVER 3.0, the tool was applied for the analysis of molecular dynamics simulation of the microbial enzyme haloalkane dehalogenase DhaA. CAVER 3.0 safely identified and reliably estimated the importance of all previously published DhaA tunnels, including the tunnels closed in DhaA crystal structures. Obtained results clearly demonstrate that analysis of molecular dynamics simulation is essential for the estimation of pathway characteristics and elucidation of the structural basis of the tunnel gating. CAVER 3.0 paves the way for the study of important biochemical phenomena in the area of molecular transport, molecular recognition and enzymatic catalysis. The software is freely available as a multiplatform command-line application at http://www.caver.cz.

In this paper, we propose a new method which considers the movement of a protein molecule as a whole for the computation of so called dynamic channels in a molecular dynamics trajectory. The method is based on maximizing the information about the empty space over time and is built on basic computational geometry principles. The dynamic channels highlight pulsing and flexible parts of the molecule. It is believed that such parts allow a ligand to pass into or out from the active site. The method was tested on real protein data and the results indicate that it presents new information about the molecule.

Collision-free paths in the geometric model of a protein molecule reveal various dependencies between the structure of the molecule and its function. The paths which connect a biochemically important part of the protein molecule with the surface of the molecule can serve as egression or access paths for small molecules which react in the active site. The geometric method introduced in this paper is designed to compute such paths in the dynamic models of protein molecules. The paths have to satisfy additional constraints such as valid flow of time which allows us to distinguish between access and egression paths, minimum width and others. Possibly, the method may be used not only for protein molecules but also for similar environments with high density of spherical obstacles. The method was tested on real protein data and the results indicate that if there is a path present, it is detected by our method.