Analyses

Beside the Analysis class serving as parent, the following analyses are implemented:

ImageAnalysis (Fit many images)
ResultParameter (Describes how the plots should by styled)
SSDAnalysis (Analyses peaks in the SSD data from a single file)
SSDAnalysisWrapper (Analyses peaks in the SSD data from many files)
PMTAnalysis (Analyses the PMT data)

class data_eng_utokyo.analyses.Analysis(recorder: Recorder, name: str, result_param: ResultParameter)[source]

Bases: object

Class for analyzing relationsships between experimental parameters and measurements.

Parameters:

recorder (Recorder)
name (str)
result_param (ResultParameter)

run()[source]

is_up_to_date()[source]

_save_results(new_result_df: DataFrame)[source]

Create if not exists else append.

Parameters:: new_result_df (DataFrame)

abstract _query_df()[source]

Narrows down the rows on which we want to perform the analysis. Example: Only select rows in certain time range. The extra parameters can be designed as child class members.

Parameters:: df (DataFrame)

abstract _run_analysis(df: DataFrame)[source]

Methods which executes the plotting and saving.

Parameters:: df (DataFrame)

_plot_2d_hist(df: DataFrame, x_column: str, y_column: str, bin_nr: int = 100, title_addition: str = '')[source]

Plots the data from the recorder as 2D histogram.

Parameters:

df (DataFrame)
x_column (str)
y_column (str)
bin_nr (int)
title_addition (str)

class data_eng_utokyo.analyses.ResultParameter(image_src: str, image_extension: str, result_filepath: str)[source]

Bases: object

Specifies how the result of the analysis should be saved.

Parameters:

image_src (str)
image_extension (str)
result_filepath (str)

class data_eng_utokyo.analyses.SSDAnalysis(recorder: SSDRecorder, result_param: ResultParameter)[source]

Bases: Analysis

Analyses the SSD data provided by a SSRecorder or SSDParser.

Example

from data_eng_utokyo.recorders import SSDRecorder
from data_eng_utokyo.analyses import SSDAnalysis
from data_eng_utokyo.analyses import ResultParameter

ssd_recorder = SSDRecorder(...)
result_param = ResultParamter(...)
ssd_analysis = SSDAnalysis(ssd_recorder, result_param)
ssd_analysis.run()

Parameters:

recorder (SSDRecorder)
result_param (ResultParameter)

_update_result_df(new_result_df)[source]: Updates the result table adding new peaks and results the part which

class data_eng_utokyo.analyses.SSDAnalysisWrapper(folder: str = '', result_path: str = '', plot_path: str = '', image_extension: str = '', match: str = '', time_interval: tuple = (datetime.datetime(2000, 1, 1, 12, 0), datetime.datetime(2030, 1, 1, 12, 0)))[source]

Bases: object

Wrapper for the SSD Analysis.

Solves the problem that we have many ssd csv file and not just one. Keeps track of the csv files that match a certain pattern in a certain timespan and applies the SSDAnalysis on them 1 by 1.

Implements the same public methods as the Analysis class, such that it can be used in Runner.

Example

ssd_wrapper = SSDAnalysisWrapper(
folder="data/sample/",
result_path="results/20220314/"+"ssd_analysis_results.csv",
plot_path="plots/20220829/",
image_extension=".png",
match=".*Slot.*.csv",
time_interval=(
    dt.datetime(2000, 1, 1, 12, 0, 0),
    dt.datetime(2030, 1, 1, 12, 0, 0)
))

Parameters:

folder (str)
result_path (str)
plot_path (str)
image_extension (str)
match (str)
time_interval (tuple)

run()[source]

is_up_to_date()[source]

class data_eng_utokyo.analyses.ImageAnalysis(recorder: FileRecorder, perform_analysis: callable, result_param: ResultParameter, time_interval: tuple | None = None, min_signal: int = 0)[source]

Bases: Analysis

Takes a recorder with image files and an analysis function and creates plots and reports.

Parameters:

recorder (Recorder) – Recorder which contains the filepaths of the images.
perform_analysis (callable) – Analysis function that is applied on each image.
result_param (ResultRecorder) – Object which tells how the plot should be formated.
time_interval (tuple) – Tuple of start and endtime. Just files in this interval will be processed.
min_signal (int) – Images with sum of pixels less than this threshold will be ignored.

Example

from data_eng_utokyo.recorder import FileRecorder
from data_eng_utokyo.analyses import MOTMLE, ResultParameter, ImageAnalysis
from data_eng_utokyo.constants import c_ccd

perform_analysis = MOTMLE(
    c=c_ccd,
    references=[],
    do_subtract_dead_pixels=False,
).perform_analysis

result_param = ResultParameter(
    image_src="../../../plots/20220829/image/",
    image_extension=".png",
    result_filepath="../../results/20220829/"+"image_analysis_results.csv",
)
file_recorder = FileRecorder(
    filepath="C:\Users\roman\Desktop\Research_UTokyo\Data\mot\",
    match=".*ccd_.*.xlsx",
)
image_analysis = ImageAnalysis(
    recorder=file_recorder,
    perform_analysis=perform_analysis,
    result_param=result_param,
    )
enriched_df = image_analysis.run()

self.was_run_before[source]

Flag.

Type:: bool

is_up_to_date()[source]

Tells if the analysis was already run with the most recent data available.

In the beginning, the recorder might be up-to-date, but the analysis was not run yet. This method helps us deal with such cases.

Returns:: Bool whether the analysis is already up to date with the data or not.

_query_df()[source]

Gets the rows of the dataframe which have a time in the interval provided in self.time_interval.

Returns:: Sliced pandas dataframe.
Parameters:: df (DataFrame)

_run_analysis(df: DataFrame)[source]

Runs the fit_mot_number algorithm on all images, saves the images and saves the fit result in a new table.

Parameters:: df (DataFrame) – Dataframe with the data as provided by the recorder.
Returns:: Enriched dataframe with additional columns representing the results of the analysis.

_enrich_df_with_statistics(df: DataFrame, statistics_list: list) → DataFrame[source]

Add the statistics generated in the analysis to the dataframe.

Takes the table from the ImageRecorder and the statistics generated by the fit_mot_data function. Combines the two in an enriched dataframe.

Parameters:

df (pd.DataFrame) – Initial dataframe as provided by the recorder.
statistics_list (list) – List of results (statistics lookups). Each item has a 1:1 corresponding to a row of the df.

Returns:

Dataframe enriched with the statistics.

Return type:

DataFrame

class data_eng_utokyo.analyses.PMTAnalysis(recorder: PMTRecorder, result_param: ResultParameter)[source]

Bases: Analysis

Parameters:

recorder (PMTRecorder)
result_param (ResultParameter)

class data_eng_utokyo.analyses.SSDHistogramAnalysis(filepath: str, image_src: str = '../../plots/', image_extension: str = '.png')[source]

Bases: Analysis

Parameters:

filepath (str)
image_src (str)
image_extension (str)