There’s a new feature in yt 4.1 that lets you load data using just functions, without it needing to be in memory or to have its own frontend defined.
Back in the long, long distant past, the Stream frontend was created to make it easy to bring data into yt without writing a full-on “frontend.” Originally this was to make it easy to share data between VTK and yt – and it worked!
Building on our ability to read data using just functions, we can now load data from raw HDF5 files with a minimum of metadata.
I’ve heard it said that HDF5 isn’t exactly a file-format. Sure, it describes how to write bits down (and does this extremely well and thoroughly), but I have always personally found it to be more immediately useful as a filesystem for data. And, it seems that people who write data to disk find it to be similarly useful – but there’s no single way that people organize the data they use HDF5 to write to disk, attempts at metadata notwithstanding.
This is a long blog post! The short of it is:
If you’re using Enzo or FLASH, you can probably do most of what you want to do with 3.0. But there are probably bugs, and you can’t volume render yet. But every bug or missing feature you find is a useful piece of information that can help speed up development. If you’re using RAMSES, 3.0 will be a vast improvement!
We’re proud to announce the release of version 2.4 of the yt Project, http://yt-project.org/ . The new version includes many new features, refinements of existing features and numerous bugfixes. We encourage all users to upgrade to take advantage of the changes.
yt is a community-developed analysis and visualization toolkit, primarily directed at astrophysical hydrodynamics simulations. It provides full support for output from the Enzo, FLASH, Orion, and Nyx codes, with preliminary support for several others.
Now that the post-workshop preparations and work have settled down, I thought it might be interesting to share some of the developments going on with yt. We’re still a long way from a new release, so these interim ‘development’ updates are meant to be a bit of a teaser. As always, these features are either in the main branch or (if noted) in a public fork on BitBucket. If they sound interesting, drop us a line on `yt-dev <http://lists.
Just in time for the New Year, we’re happy to announce the release of yt version 2.3! ( http://yt-project.org/ ) The new version includes many new modules and enhancements, and the usual set of bug fixes over the last point release. We encourage all users to upgrade to take advantage of the changes.
yt is a community-developed analysis and visualization toolkit for astrophysical simulation data. yt provides full support for Enzo, Orion, Nyx, and FLASH codes, with preliminary support for the RAMSES code (and a handful of others.
A few of us worked this past week on a couple yt projects and made what we think is significant progress. Two of the items we focused on were testing and parallelism.
For testing, we’ve broadened the test suite to include many more functions and derived quantities. We now have 548 tests that include (off and on-axis) slices, (off and on- axis) projections, phase distributions, halo finding, volume rendering, and geometrical region cuts such as rectangular solids, spheres, and disks.
A few of the yt developers have been experimenting with screencasts to show off new features or demonstrate how to do some things. Sam and I have both prepared screencasts on volume rendering and getting started with developing, respectively. Check them out below, and please feel free to leave comments and let us know what you think – not just about the screencasts, but about what they demonstrate, and if you think any of the concepts or routines could be made easier.
In this post I’d like to discuss a bit of work in progress to highlight some exciting new features that we hope to have working in yt sometime soon.
On any machine that runs yt, there is a file created in the users home directory named ~/.yt/parameter_files.csv that yt uses internally to keep track of datasets it has seen. This is just a simple text file containing comma-separated entries with a few pieces of information about datasets, like their location on disk and the last date and time they were ‘seen’ by yt.
This last week, following the release of version 2.2 of yt, I spent a bit of time looking at speed improvements. There were several places that the code was unacceptably slow:
1D profiles (as noted in our method paper, even) Ghost-zone generation RAMSES grid data loading The first of these was relatively easy to fix. In the past, 1D profiles (unlike 2D profiles) were calculated using pure-python mechanisms; numpy was used for digitization, then inverse binning was conducted by the numpy ‘where’ command, and these binnings were used to generate the overall histogram.
Thanks to some awesome work by developers at Kitware, yt can now be called from within ParaView!
This relies on a couple things, all of which are pretty exciting moving forward. The idea behind this is to leverage where the two codes have their own strengths and weaknesses, and identify places where they can work productively together. For instance, yt has been designed to provide astrophysical analysis; in this way, it can generate adaptive projections, spectral energy distributions and so on.
It’s been a while since the last Development post — but in that time, some pretty fun things have been going on. These are some of the smaller things, but there are bigger things in store which I’ll write about next time.
QuadTree Projections For a while, there has been the option to use a QuadTree data structure to conduct projections through a simulation volume. This was accessible through the quad_proj attribute, which respected the same interface as the normal proj interface.
We are proud to announce the release of yt version 2.1. This release includes several new features, bug fixes, and numerous improvements to the code base and documentation. At the yt homepage, http://yt.enzotools.org/ , an installation script, a cookbook, documentation and a guide to getting involved can be found.
yt is an analysis and visualization toolkit for Adaptive Mesh Refinement data. yt provides full support for Enzo, Orion, and FLASH codes, with preliminary support for RAMSES, ART, Chombo, CASTRO and MAESTRO codes.
It’s been nearly a month since the last yt development post; in that time, there’s been quite a bit of development in a couple different areas. This is culminating in a 2.1 release, for which Sam Skillman is release manager, sometime in the next few days.
Streamlines and Treecode SamS has spent some time over the last month developing two types of streamline code. The first integrates a series of streamlines over a selection of the domain, which can then be visualizing using the mplot3d package.
This last week was the first full week on BitBucket and so far I think it has been quite successful. The new development process is for most of the core developers to maintain personal forks for experimental changes, or longer term changes, and then to commit directly or merge when bug fixes or features are ready to be integrated. The list of forks is easily visible and each individual fork’s divergence from the primary repository can be viewed by clicking on the green arrows.
The major changes this week came mostly in the form of administrative shifts. However, SamS did some great work I’m going to hint at (he’ll post a blog entry later) and I started laying the ground work for something I’ve been excited about for a while, an MPI-aware task queue.
BitBucket For the last couple months, yt has been struggling under the constraints of the hg server on its hosting plan.
This week there was not very much yt development. However, a few notes may be of interest. SamS has updated the HEALPix camera to support ordered projections; what this means is that you can now make volume renderings using a standard color transfer function, or even the Planck transfer function, that cover 4pi of the sky. I am still working on integrating a method for creating images easily, but for now the scripts from last week should work.
yt has many extension packages to help you in your scientific workflow! Check
these out, or create your own.
ytini is set of tools and tutorials for using yt as a tool inside the 3D visual
effects software Houdini or a data pre-processor externally to Houdini.
Trident is a full-featured tool that projects arbitrary sightlines through
astrophysical hydrodynamics simulations for generating mock spectral
observations of the IGM and CGM.
pyXSIM is a Python package for simulating X-ray observations from astrophysical
Analyze merger tree data from multiple sources. It’s yt for merger trees!
yt_idv is a package for interactive volume rendering with yt!
It provides interactive visualization using OpenGL for datasets
loaded in yt. It is written to provide both scripting and interactive access.
widgyts is a jupyter widgets extension for yt,
backed by rust/webassembly to allow for browser-based, interactive exploration
of data from yt.
We welcome contributions from all members of the yt community. Feel free to
reach out if you need any help.
the yt data hub
The yt hub at https://girder.hub.yt/ has a ton of resources to check out,
whether you have yt installed or not.
The collections host all sorts of data
that can be loaded with yt. Some have been used in publications, and others are
used as sample frontend data for yt. Maybe there’s data from your simulation
The rafts host the yt quickstart notebooks,
where you can interact with yt in the browser, without needing to install it
locally. Check out some of the other rafts too, like the widgyts release
notebooks – a demo of the widgyts yt extension pacakge; or the notebooks from
the CCA workshop – a user’s workshop on using yt.