{ "cells": [ { "cell_type": "markdown", "id": "9acfb438-b43c-4042-a9eb-982da0528bff", "metadata": {}, "source": [ "# Edit an NXtomo\n", "\n", "The general workflow to edit an NXtomo is:\n", "\n", "```\n", "load it from disk -> modify it in memory -> save it to disk\n", "```\n", "\n", "In this example we edit the `dummy_nxtomo.nx` file, produced by the \"Create an NXtomo (from scratch)\" tutorial.\n" ] }, { "cell_type": "code", "execution_count": null, "id": "4ccc9c2c-cacf-423d-aa44-c8958c9b8f57", "metadata": {}, "outputs": [], "source": [ "import os\n", "from nxtomo import NXtomo\n", "\n", "nx_tomo_file_path = os.path.join(\"resources\", \"dummy_nxtomo.nx\")\n", "nx_tomo = NXtomo().load(nx_tomo_file_path, \"entry\", detector_data_as=\"as_numpy_array\")\n", "print(\"nx_tomo type is\", type(nx_tomo))\n", "print(\"nx_tomo energy is\", nx_tomo.energy)" ] }, { "cell_type": "markdown", "id": "5978b2db-900f-4b46-ae6b-28d8d4c01958", "metadata": {}, "source": [ "You can then modify the values as shown previously and overwrite the file.\n" ] }, { "cell_type": "code", "execution_count": null, "id": "b35f9b9b-d80d-4b51-814c-bb829d7d6b94", "metadata": {}, "outputs": [], "source": [ "import pint\n", "\n", "ureg = pint.UnitRegistry()\n", "\n", "nx_tomo.energy = 13.6 * ureg.keV\n", "nx_tomo.save(\n", " file_path=nx_tomo_file_path,\n", " data_path=\"entry\",\n", " overwrite=True,\n", ")\n", "print(\"new energy is\", NXtomo().load(nx_tomo_file_path, \"entry\").energy)" ] }, { "cell_type": "markdown", "id": "2c6ace38-4ab9-4efb-961c-ec7afb6b4751", "metadata": {}, "source": [ "
\n", "
\n", " \n", "
\n", " Detector data is usually stored as an h5py virtual dataset. Large acquisitions can consume significant memory, so detector data can be loaded with several strategies:\n", "
    \n", "
  • \"as_data_url\" (default): each VirtualSource is saved as a DataUrl to keep handling lightweight (see later in the tutorial).
    • \n", "
  • \"as_virtual_source\": retrieves the original VirtualSource objects so you can edit them.
    • \n", "
  • \"as_numpy_array\": loads all frames in memory for editing and writes everything back. Avoid this for real datasets as it triggers heavy I/O.
    • \n", "
\n", "
\n", "
\n", "
\n" ] }, { "cell_type": "markdown", "id": "55a76aa3-6053-4682-ac5c-e08fc4e5bd55", "metadata": {}, "source": [ "#### Clean\n" ] }, { "cell_type": "code", "execution_count": null, "id": "8395a7cf-6f41-4fb6-9c7a-bb181f7aa32d", "metadata": {}, "outputs": [], "source": [ "if os.path.exists(nx_tomo_file_path):\n", " os.remove(nx_tomo_file_path)\n", "if os.path.exists(\"nxtomo_reconstruction.hdf5\"):\n", " os.remove(\"nxtomo_reconstruction.hdf5\")" ] }, { "cell_type": "markdown", "id": "ab38514e-ed51-4760-8786-080fb546622a", "metadata": {}, "source": [ "## Advanced usage: provide DataUrl objects to instrument.detector.data\n", "The NXtomo described above can consume lots of memory when the detector is large or the number of projections is high.\n", "\n", "A practical workaround is to supply DataUrl objects that may point to external files and let the FrameAppender manage them.\n", "\n", "In this example we first store the metadata in the HDF5 file (and optionally a few frames). You can then append frame series sequentially with their rotation angles and image keys.\n" ] }, { "cell_type": "markdown", "id": "3bf3b4af-0fcd-4cd4-8482-ac191c4db45a", "metadata": {}, "source": [ "### Create datasets in external files\n", "\n", "Here we create datasets in external files and record the corresponding DataUrls.\n" ] }, { "cell_type": "markdown", "id": "179b77d7-50bd-4656-a93f-94d6384cf7bd", "metadata": {}, "source": [ "
\n", " \n", "
\n", " These datasets must be 3D; otherwise the virtual dataset creation will fail.\n", "
\n", "
\n" ] }, { "cell_type": "code", "execution_count": null, "id": "b726786a-93de-4ae1-82b9-82ba7da3a5da", "metadata": {}, "outputs": [], "source": [ "import h5py\n", "import numpy\n", "from silx.io.url import DataUrl\n", "\n", "detector_data_urls = []\n", "for i_file in range(5):\n", " os.makedirs(\"output/external_files\", exist_ok=True)\n", " external_file = os.path.join(f\"output/external_files/file_{i_file}.nx\")\n", " with h5py.File(external_file, mode=\"w\") as h5f:\n", " h5f[\"data\"] = numpy.arange(\n", " start=(5 * 100 * 100 * i_file), stop=(5 * 100 * 100 * (i_file + 1))\n", " ).reshape(\n", " [5, 100, 100]\n", " ) # of course here this is most likely that you will load data from another file\n", "\n", " detector_data_urls.append(\n", " DataUrl(\n", " file_path=external_file,\n", " data_path=\"data\",\n", " scheme=\"silx\",\n", " )\n", " )" ] }, { "cell_type": "markdown", "id": "3f8200bb-ae51-419c-b8db-13690835e6c9", "metadata": {}, "source": [ "### Create a simple NXtomo and provide DataUrl entries for instrument.detector.data\n" ] }, { "cell_type": "code", "execution_count": null, "id": "0191a47b-10a5-4af3-b639-1608e0fd2c29", "metadata": {}, "outputs": [], "source": [ "my_large_nxtomo = NXtomo()" ] }, { "cell_type": "markdown", "id": "9df7b9a8-e5e4-4ffe-bebf-e83cf5573de3", "metadata": {}, "source": [ "Provide all metadata except the frames. In this example we create a dataset with 180 projections.\n" ] }, { "cell_type": "code", "execution_count": null, "id": "d219ec05-b8a4-49c9-add2-cc4f15e4f7df", "metadata": {}, "outputs": [], "source": [ "my_large_nxtomo.instrument.detector.distance = 0.2 * ureg.meter\n", "my_large_nxtomo.instrument.detector.x_pixel_size = (\n", " my_large_nxtomo.instrument.detector.y_pixel_size\n", ") = (12 * ureg.micrometer)\n", "my_large_nxtomo.energy = 12.3 * ureg.keV\n", "# ...\n", "my_large_nxtomo.sample.rotation_angle = (\n", " numpy.linspace(0, 180, 180, endpoint=False) * ureg.degree\n", ")\n", "my_large_nxtomo.instrument.detector.image_key_control = [0] * 180 # 0 == Projection" ] }, { "cell_type": "markdown", "id": "c4e5f8fc-ca0c-47dc-a0e1-1067fa1e2a1f", "metadata": {}, "source": [ "Provide the list of DataUrl objects to `instrument.detector.data`.\n" ] }, { "cell_type": "code", "execution_count": null, "id": "cda2cbed-2b0d-48d5-bab8-272fc8e0b80c", "metadata": {}, "outputs": [], "source": [ "my_large_nxtomo.instrument.detector.data = detector_data_urls" ] }, { "cell_type": "code", "execution_count": null, "id": "6c853847-0158-4fc0-82d6-2eb01129d043", "metadata": {}, "outputs": [], "source": [ "os.makedirs(\"output\", exist_ok=True)\n", "my_large_nxtomo.save(\"output/my_large_nxtomo.nx\", data_path=\"entry0000\", overwrite=True)" ] }, { "cell_type": "markdown", "id": "559875fd-637e-46df-86e0-7d442da5e50d", "metadata": {}, "source": [ "
\n", " \n", "
\n", " This creates a virtual dataset under 'instrument/detector/data' that stores relative links from \"my_large_nxtomo.nx\" to the external files.\n", "
\n", "
\n" ] }, { "cell_type": "markdown", "id": "bfae56a3-60a0-4d07-b7ec-1a505e3d1bcd", "metadata": {}, "source": [ "The `data` dataset now contains 180 frames (running the previous cell again keeps appending data).\n", "\n", "If a DataUrl is provided instead of a NumPy array, NXtomo builds a virtual dataset and avoids duplicating data. Keep the files in the same relative locations.\n", "\n", "**Appended frames must have the same dimensions; otherwise the operation fails.**\n" ] }, { "cell_type": "code", "execution_count": null, "id": "5bd78498-59d3-45b9-88b9-7c9a80346f30", "metadata": {}, "outputs": [], "source": [ "from h5glance import H5Glance\n", "\n", "H5Glance(\"output/my_large_nxtomo.nx\")" ] }, { "cell_type": "markdown", "id": "f9e885eb-111e-40b8-9655-7e1a7ef391f2", "metadata": {}, "source": [ "Check that the paths of the VirtualSource objects are relative (they must start with './').\n" ] }, { "cell_type": "code", "execution_count": null, "id": "9ca2c9d9-5881-4fa6-ac39-47394d44d7b3", "metadata": { "tags": [] }, "outputs": [], "source": [ "with h5py.File(\"output/my_large_nxtomo.nx\", mode=\"r\") as h5f:\n", " dataset = h5f[\"entry0000/instrument/detector/data\"]\n", " print(\"dataset is virtual:\", dataset.is_virtual)\n", " for vs_info in dataset.virtual_sources():\n", " print(\"file name is\", vs_info.file_name)\n", " assert vs_info.file_name.startswith(\"./\")" ] }, { "cell_type": "markdown", "id": "561b423a-3e00-4004-a5b9-7d9a474ba6d1", "metadata": {}, "source": [ "
\n", " \n", "
\n", " tip
\n", " \n", "
\n", "
\n" ] }, { "cell_type": "markdown", "id": "4f0b1b10-b3b8-488f-89f5-24fae10f91d5", "metadata": {}, "source": [ "## Advanced use cases" ] }, { "cell_type": "markdown", "id": "fd246a40-88a0-452d-b327-52145b9fddde", "metadata": {}, "source": [ "### Provide NXtransformations to NXdetector (detector flip, rotation, translation...)\n", "\n", "Detectors may include image flips (up-down or left-right) introduced by the acquisition (BLISS-Tango) or manual adjustments such as rotations around an axis.\n", "\n", "To describe them, `NXdetector` exposes a `TRANSFORMATIONS` group that defines the transformation chain.\n", "As of today (2023) only image flips are taken into account by nabu for stitching.\n", "\n", "To provide such transformations you can supply a set of transformation entries like the following:\n" ] }, { "cell_type": "code", "execution_count": null, "id": "f77ede82-0c75-4a8f-b428-3813f8f389d2", "metadata": { "tags": [] }, "outputs": [], "source": [ "from nxtomo import NXtomo\n", "\n", "my_nxtomo = NXtomo()" ] }, { "cell_type": "code", "execution_count": null, "id": "5fd411dc-f337-4c50-8231-c0d73b01fef0", "metadata": { "tags": [] }, "outputs": [], "source": [ "import pint\n", "from nxtomo.utils.transformation import Transformation\n", "\n", "_ureg = pint.UnitRegistry()\n", "\n", "my_nxtomo.instrument.detector.transformations.add_transformation(\n", " Transformation(\n", " axis_name=\"rx\", # axis name must be unique\n", " transformation_type=\"rotation\",\n", " value=180 * ureg.degree,\n", " vector=(\n", " 1,\n", " 0,\n", " 0,\n", " ), # warning: transformation are provided as (x, y, z) which is different of the usual numpy ref used (z, y, x)\n", " )\n", ")" ] }, { "cell_type": "markdown", "id": "aaa28d86-8a32-4f8a-a073-90e857ed6fb3", "metadata": {}, "source": [ "There are several utility classes that provide detector flips and basic transformation axes for you.\n", "Please consider using them.\n" ] }, { "cell_type": "code", "execution_count": null, "id": "77832746-948b-4e97-8fab-4bb8e71c607a", "metadata": { "tags": [] }, "outputs": [], "source": [ "from nxtomo.utils.transformation import (\n", " Transformation,\n", " DetYFlipTransformation,\n", " DetZFlipTransformation,\n", " TransformationAxis,\n", " TransformationType,\n", ")\n", "from nxtomo.nxobject.nxtransformations import NXtransformations\n", "\n", "nx_transformations = NXtransformations()\n", "nx_transformations.transformations = (\n", " DetYFlipTransformation(flip=True), # vertical flip of the detector\n", " DetZFlipTransformation(\n", " flip=True, depends_on=\"ry\"\n", " ), # horizontal flip of the detector. Applied after the vertical flip\n", " Transformation( # some translation over x axis. Applied after the horizontal flip\n", " axis_name=\"tx\",\n", " value=0.02\n", " * ureg.millimeter, # value can be a scalar - static value - of an array of value (one per frame expected)\n", " transformation_type=TransformationType.TRANSLATION, # default unit for translation is SI 'meter'\n", " depends_on=\"rz\", # Applied after the horizontal flip in the transformation chain\n", " vector=TransformationAxis.AXIS_X,\n", " ),\n", ")\n", "my_nxtomo.instrument.detector.transformations = nx_transformations" ] }, { "cell_type": "markdown", "id": "e818a401-a3ee-472a-adb2-0f2e11c1d3cb", "metadata": {}, "source": [ "
\n", " \n", " NXtransformations and NXsample\n", "
\n", " As of today the NXtomo application uses individual datasets (`x_translation`, `y_translation`, `z_translation`, `rotation_angle`) to describe sample transformations. Future versions may adopt an NXtransformations group, but this is not yet implemented, so keep using the original datasets for transformations.\n", "
\n", "
\n" ] }, { "cell_type": "code", "execution_count": null, "id": "0be99b25-e62e-4194-a10a-e935eeaf8b68", "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": ".venv", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.11" } }, "nbformat": 4, "nbformat_minor": 5 }