yt development - External Analysis and Simulation Code Support
@ Matthew Turk | Monday, Mar 7, 2011 | 3 minute read | Update at Monday, Mar 7, 2011

This last week was the first full week on BitBucket and so far I think it has been quite successful. The new development process is for most of the core developers to maintain personal forks for experimental changes, or longer term changes, and then to commit directly or merge when bug fixes or features are ready to be integrated. The list of forks is easily visible and each individual fork’s divergence from the primary repository can be viewed by clicking on the green arrows. All of the new mechanisms for developing using BitBucket are included in the ‘How to Develop yt’ section of the documentation.

This last week I spent a few days at KITP’s Galaxy Clusters workshop, where I presented on yt. There were a few major points that came out of my visit, talking to the simulators there, that are germane to the long term development of yt.

  • As time goes on, yt should be increasingly viewed as a mechanism not just for analyzing data with its own, internal analysis routines, but as a mechanism for handling data, transforming it into a uniform interface independent of the underlying simulation code. This will allow for linking against and utilizing external analysis codes much more easily. (The three examples that came up while I was at KITP were a new halo finder, a weak lensing code, and a radiation transport code.) To facilitate the process of calling external codes from within yt, I’ve written a section in the documentation that covers it.
  • There is a great deal of interest in ensuring yt works equally well with many different simulation platforms. This is a primary goal of my current fellowship, and I am working toward it. The next two codes that will be targeted for improvement are Gadget and ART, and I made good contacts at the workshop to this end.
  • The idea of analysis modules, particularly in a block-programming environment, is compelling. There is quite a bit of interest in an interface where inputs and outputs were handled like pipes. I am still formulating my ideas on this. Last Fall I experimented a bit with an introspection system that could handle arguments and could hook up pipes, but it never got very far. (The code.)

I had a number of scientific takeaways from the meeting, too, but that would all go into a different blog post.

This week I hope to finish up the adaptive ray tracing. This past week StephenS unveiled a halo serialization mechanism, which I Think many people are excited about, and SamS continued developing his streamline code.

yt extension modules

yt has many extension packages to help you in your scientific workflow! Check these out, or create your own.

ytini

ytini is set of tools and tutorials for using yt as a tool inside the 3D visual effects software Houdini or a data pre-processor externally to Houdini.

Trident

Trident is a full-featured tool that projects arbitrary sightlines through astrophysical hydrodynamics simulations for generating mock spectral observations of the IGM and CGM.

pyXSIM

pyXSIM is a Python package for simulating X-ray observations from astrophysical sources.

ytree

Analyze merger tree data from multiple sources. It’s yt for merger trees!

yt_idv

yt_idv is a package for interactive volume rendering with yt! It provides interactive visualization using OpenGL for datasets loaded in yt. It is written to provide both scripting and interactive access.

widgyts

widgyts is a jupyter widgets extension for yt, backed by rust/webassembly to allow for browser-based, interactive exploration of data from yt.

yt_astro_analysis

yt_astro_analysis is the yt extension package for astrophysical analysis.

Make your own!!

Finally, check out our development docs on writing your own yt extensions!

Contributing to the Blog

Are you interested in contributing to the yt blog?

Check out our post on contributing to the blog for a guide!

We welcome contributions from all members of the yt community. Feel free to reach out if you need any help.

the yt data hub

The yt hub at https://girder.hub.yt/ has a ton of resources to check out, whether you have yt installed or not.

The collections host all sorts of data that can be loaded with yt. Some have been used in publications, and others are used as sample frontend data for yt. Maybe there’s data from your simulation software?

The rafts host the yt quickstart notebooks, where you can interact with yt in the browser, without needing to install it locally. Check out some of the other rafts too, like the widgyts release notebooks – a demo of the widgyts yt extension pacakge; or the notebooks from the CCA workshop – a user’s workshop on using yt.

Social Links