Individual datasets¶

In this chapter we'll look at what we can do with an individual dataset. In hoa-tools individual datasets are represented by Dataset objects.

In [1]:

import pandas as pd

import pandas as pd

In [2]:

import hoa_tools.dataset
import hoa_tools.inventory

import hoa_tools.dataset import hoa_tools.inventory

In [3]:

pd.options.display.width = None  # type: ignore[assignment]

pd.options.display.width = None # type: ignore[assignment]

First, lets print all the spleen datasets available from the inventory::

In [4]:

inventory = hoa_tools.inventory.load_inventory()
spleen_inventory = inventory[inventory["organ"] == "spleen"]
spleen_inventory

inventory = hoa_tools.inventory.load_inventory() spleen_inventory = inventory[inventory["organ"] == "spleen"] spleen_inventory

Out[4]:

	donor	organ	organ_context	voi	voxel_size_um
LADAF-2022-16_spleen_complete-organ_15.75um_bm18	LADAF-2022-16	spleen	NaN	complete-organ	15.75
LADAF-2020-27_spleen_central-column_1.29um_bm05	LADAF-2020-27	spleen	NaN	central-column	1.29
LADAF-2020-27_spleen_complete-organ_25.08um_bm05	LADAF-2020-27	spleen	NaN	complete-organ	25.08
LADAF-2021-17_spleen_complete-organ_25.0um_bm05	LADAF-2021-17	spleen	NaN	complete-organ	25.00
LADAF-2020-27_spleen_central-column_6.05um_bm05	LADAF-2020-27	spleen	NaN	central-column	6.05

We can see that three datasets are available. They are from the same organ, with one being the full organ dataset and two being high resolution region of interest datasets. Lets get the full organ dataset

In [5]:

whole_spleen = hoa_tools.dataset.get_dataset(
    "LADAF-2020-27_spleen_complete-organ_25.08um_bm05"
)
whole_spleen

whole_spleen = hoa_tools.dataset.get_dataset( "LADAF-2020-27_spleen_complete-organ_25.08um_bm05" ) whole_spleen

Out[5]:

Dataset(name='LADAF-2020-27_spleen_complete-organ_25.08um_bm05', dataset_type='overview', voi='complete-organ', data=Data(shape=[2919, 2151, 1900], voxel_size_um=25.08, gcs_url='n5://gs://ucl-hip-ct-35a68e99feaae8932b1d44da0358940b/LADAF-2020-27/spleen/25.08um_complete-organ_bm05/'), sample=Sample(organ='spleen', organ_context=None, post_mortem_interval_hours=None, fixation_method=None, fixation_medium='formalin', organ_infilled=None, stabilisation_medium='crushed agar', degassing_method='vacuum', scan_solvent='ethanol', scan_solvent_concentration=70.0, scan_temperature='room temperature'), donor=Donor(id='LADAF-2020-27', age=Age(root=94), sex='F', weight=Weight(root=45.0), height=Height(root=140.0), cause_of_death=None, medical_history='right sylvian and right cerebellar stroke, Cognitive disorders of vascular origin, Depressive syndrome, Rhythmic and hypertensive heart disease (AF), Micro-crystalline arthritis (gout), Right lung pneumopathy (3 years before death), Left cataract, Left temporal squamous cell carcinoma.', diabetes=None, hypertension='Yes', smoker=None), scan=Scan(date=datetime.date(2020, 7, 12), beamline='BM05', energy=Energy(root=93.0), current_start=CurrentStart(root=200.0), filling_mode=None, n_frames=NFrames(root=6000), n_ref=NRef(root=0), n_dark=NDark(root=400), latency_time=LatencyTime(root=0.004), exposure_time=ExposureTime(root=0.4), subframe_time=SubframeTime(root=0.04), n_subframes=NSubframes(root=10), scan_type='zseries', scan_range=360.0, n_scans=NScans(root=25), acquisition='half', z_step=2.0, scan_time=ScanTime(root=300.0), filters=['Mo 0.1mm', 'Al 2mm', 'SiO2 60mm rods 12x5mm'], scintillator='LuAG:Ce 2000um', optic='DZoom', optic_magnification=OpticMagnification(root=0.24), distance_source_sample=None, distance_sample_detector=DistanceSampleDetector(root=3475.0), psho=None, sensor_name='PCO edge 4.2 CLHS', sensor_mode='rolling shutter', sensor_roi_x_size=SensorRoiXSize(root=176), sensor_roi_y_size=SensorRoiYSize(root=176), sensor_binning=None, pixel_size=25.08, xray_magnification=None, technique='Hierarchical Phase-Contrast Tomography (HiP-CT)', experiment_type='Tomography'), proposal=Proposal(proposal_number='md1252', title="Multiscale Quantification of Covid-19's impact on lung vasculature from whole lobe to alveolar/microvascular scales", proposers=['Peter Lee', 'Paul Tafforeau', 'Danny Jonigk', 'Maximilian Ackermann', 'Mark P. Kuhnel', 'Elodie Boller', 'Willi Wagner']), registration=None, citation=Citation(title='Overview at 25.08um of the spleen of donor LADAF-2020-27, scanned at ESRF on beamline BM05.', contributors=[Contributor(first_name='The', last_name='Human Organ Atlas Collaboration', orcid=None, roles=[]), Contributor(first_name='Claire', last_name='Walsh', orcid=Orcid(root='0000-0003-3769-3392'), roles=['Conceptualization', 'Funding acquisition']), Contributor(first_name='Danny', last_name='Jonigk', orcid=Orcid(root='0000-0002-5251-2281'), roles=['Funding acquisition', 'Investigation']), Contributor(first_name='David', last_name='Stansby', orcid=Orcid(root='0000-0002-1365-1908'), roles=['Data curation', 'Software']), Contributor(first_name='Elodie', last_name='Boller', orcid=None, roles=['Funding acquisition', 'Investigation']), Contributor(first_name='Guillaume', last_name='Gaisne', orcid=Orcid(root='0000-0002-3401-7930'), roles=['Data curation', 'Software']), Contributor(first_name='Hector', last_name='Dejea I Velardo', orcid=Orcid(root='0000-0003-2584-9812'), roles=['Data curation', 'Investigation']), Contributor(first_name='Joanna', last_name='Purzycka', orcid=Orcid(root='0000-0001-8742-5279'), roles=['Data curation']), Contributor(first_name='Joseph', last_name='Brunet', orcid=Orcid(root='0000-0002-8424-9510'), roles=['Data curation', 'Investigation', 'Software']), Contributor(first_name='Mark', last_name='P. Kuhnel', orcid=Orcid(root='0000-0003-3558-2576'), roles=['Funding acquisition']), Contributor(first_name='Maximilian', last_name='Ackermann', orcid=Orcid(root='0000-0001-9996-2477'), roles=['Funding acquisition']), Contributor(first_name='Paul', last_name='Tafforeau', orcid=Orcid(root='0000-0002-5962-1683'), roles=['Conceptualization', 'Data curation', 'Funding acquisition', 'Investigation']), Contributor(first_name='Peter', last_name='Lee', orcid=Orcid(root='0000-0002-3898-8881'), roles=['Conceptualization', 'Funding acquisition', 'Investigation']), Contributor(first_name='Theresa', last_name='Urban', orcid=Orcid(root='0000-0002-0352-8180'), roles=['Data curation', 'Investigation', 'Software']), Contributor(first_name='Willi', last_name='Wagner', orcid=None, roles=['Funding acquisition', 'Investigation'])], doi='10.15151/ESRF-DC-572244468', author_list=['The Human Organ Atlas Collaboration', 'Claire Walsh', 'Danny Jonigk', 'David Stansby', 'Elodie Boller', 'Guillaume Gaisne', 'Hector Dejea I Velardo', 'Joanna Purzycka', 'Joseph Brunet', 'Mark P. Kuhnel', 'Maximilian Ackermann', 'Paul Tafforeau', 'Peter Lee', 'Theresa Urban', 'Willi Wagner']), metadata_version='1.1')

Because this is a full-organ dataset, it will have a number of child datasets. These are scans of the same organ taken at a higher resolution over a subset of the full volume. For our selected dataset these child datasets are::

In [6]:

child_datasets = whole_spleen.get_children()
for dataset in child_datasets:
    print(dataset)

child_datasets = whole_spleen.get_children() for dataset in child_datasets: print(dataset)

Dataset(name=LADAF-2020-27_spleen_central-column_1.29um_bm05)
Dataset(name=LADAF-2020-27_spleen_central-column_6.05um_bm05)

We can see there are two child datasets, one at a resolution of 1.29 μm and one at a resolution of 6.05 μm.