OOS Tools#

This module provides classes and functions for processing and analyzing Out-Of-Straightness (OOS) pipeline survey and route data.

Features:

Designed for use in subsea pipeline, but general enough for any OOS survey data processing and analysis tasks.
The OOSAnonymisation class processes OOS survey and route data, providing methods for cleaning, sectioning, coordinate normalization, and anonymization of pipeline survey datasets.
The OOSDespiker class implements rolling window sigma-clipping despiking for OOS survey data, removing outliers while preserving data alignment and group structure.
The FFTSmoother and GaussianSmoother class implements group-wise signal processing and smoothing (FFT, Gaussian, etc.) for OOS survey data, supporting robust, efficient, and reproducible analysis.

All calculations are vectorized using NumPy and SciPy for efficiency and flexibility.

class refpy.oos_tools.OOSAnonymisation(*, route_development, route_pipeline_group, route_pipeline, route_kp_from, route_kp_to, route_pipeline_section_type, route_curve_radius, survey_development, survey_type, survey_pipeline_group, survey_pipeline, survey_kp, survey_easting, survey_northing)[source]#

Bases: object

Processor for Out-Of-Straightness (OOS) pipeline survey and route data. Provides methods for cleaning, sectioning, and coordinate normalization. All input arrays (parameters) are converted to NumPy arrays and stored as attributes. Additional attributes are initialized as NumPy arrays and will be populated by processing methods.

Parameters:

route_development (np.ndarray) – Development identifiers for each route section.
route_pipeline_group (np.ndarray) – Pipeline group identifiers for each route section.
route_pipeline (np.ndarray) – Pipeline identifiers for each route section.
route_kp_from (np.ndarray) – Start KP (kilometer point) for each route section.
route_kp_to (np.ndarray) – End KP (kilometer point) for each route section.
route_pipeline_section_type (np.ndarray) – Section type (e.g., ‘Straight’, ‘Curve’, etc.) for each route section.
route_curve_radius (np.ndarray) – Design curve radius for each route section.
survey_development (np.ndarray) – Development identifiers for each survey point.
survey_type (np.ndarray) – Survey type for each survey point.
survey_pipeline_group (np.ndarray) – Pipeline group identifiers for each survey point.
survey_pipeline (np.ndarray) – Pipeline identifiers for each survey point.
survey_kp (np.ndarray) – KP (kilometer point) for each survey point.
survey_easting (np.ndarray) – Easting coordinate for each survey point.
survey_northing (np.ndarray) – Northing coordinate for each survey point.

process(anonymise: bool = False)[source]#

Run the OOS processing pipeline.

Parameters:: anonymise (bool, default False) – If True, runs the full anonymisation pipeline (with cleaning and grouping). If False, runs only the basic section processing.

get_route_development()[source]#

Get development identifiers for each route section.

Returns:: Development identifiers for each route section.
Return type:: np.ndarray

get_route_pipeline_group()[source]#

Get pipeline group identifiers for each route section.

Returns:: Pipeline group identifiers for each route section.
Return type:: np.ndarray

get_route_pipeline()[source]#

Get pipeline identifiers for each route section.

Returns:: Pipeline identifiers for each route section.
Return type:: np.ndarray

get_route_kp_from()[source]#

Get start KP (kilometer point) for each route section.

Returns:: Start KP (kilometer point) for each route section.
Return type:: np.ndarray

get_route_kp_to()[source]#

Get end KP (kilometer point) for each route section.

Returns:: End KP (kilometer point) for each route section.
Return type:: np.ndarray

get_route_pipeline_section_type()[source]#

Get section type (e.g., ‘Straight’, ‘Curve’, etc.) for each route section.

Returns:: Section type for each route section.
Return type:: np.ndarray

get_route_curve_radius()[source]#

Get design curve radius for each route section.

Returns:: Design curve radius for each route section.
Return type:: np.ndarray

get_survey_development()[source]#

Get development identifiers for each survey point.

Returns:: Development identifiers for each survey point.
Return type:: np.ndarray

get_survey_type()[source]#

Get survey type for each survey point.

Returns:: Survey type for each survey point.
Return type:: np.ndarray

get_survey_pipeline_group()[source]#

Get pipeline group identifiers for each survey point.

Returns:: Pipeline group identifiers for each survey point.
Return type:: np.ndarray

get_survey_pipeline()[source]#

Get pipeline identifiers for each survey point.

Returns:: Pipeline identifiers for each survey point.
Return type:: np.ndarray

get_survey_kp()[source]#

Get KP (kilometer point) for each survey point.

Returns:: KP (kilometer point) for each survey point.
Return type:: np.ndarray

get_survey_easting()[source]#

Get easting coordinate for each survey point.

Returns:: Easting coordinate for each survey point.
Return type:: np.ndarray

get_survey_northing()[source]#

Get northing coordinate for each survey point.

Returns:: Northing coordinate for each survey point.
Return type:: np.ndarray

get_survey_section_type()[source]#

Get section type (‘Straight’, ‘Curve’, etc.) for each survey point.

Returns:: Section type for each survey point.
Return type:: np.ndarray

get_survey_section_no()[source]#

Get section number for each survey point.

Returns:: Section number for each survey point.
Return type:: np.ndarray

get_survey_section_kp_mod()[source]#

Get section-modified KP for each survey point.

Returns:: Section-modified KP for each survey point.
Return type:: np.ndarray

get_survey_section_easting_mod()[source]#

Get section-modified easting coordinates for each survey point.

Returns:: Section-modified easting coordinates for each survey point.
Return type:: np.ndarray

get_survey_section_northing_mod()[source]#

Get section-modified northing coordinates for each survey point.

Returns:: Section-modified northing coordinates for each survey point.
Return type:: np.ndarray

get_survey_feature()[source]#

Get feature label (e.g., ‘PWMD’, ‘LBMD’, ‘ILT’) for each survey point.

Returns:: Feature label for each survey point.
Return type:: np.ndarray

get_survey_design_route_curve_radius()[source]#

Get design route curve radius for each survey point.

Returns:: Design route curve radius for each survey point.
Return type:: np.ndarray

get_survey_actual_route_curve_radius()[source]#

Get fitted curve radius for each survey point.

Returns:: Fitted curve radius for each survey point.
Return type:: np.ndarray

get_survey_group_section_no()[source]#

Get group section number for each survey point.

Returns:: Group section number for each survey point.
Return type:: np.ndarray

get_survey_group_section_type()[source]#

Get group section type for each survey point.

Returns:: Group section type for each survey point.
Return type:: np.ndarray

get_survey_group_section_kp_mod()[source]#

Get group section-modified KP for each survey point.

Returns:: Group section-modified KP for each survey point.
Return type:: np.ndarray

get_survey_group_section_easting_mod()[source]#

Get group section-modified easting coordinates for each survey point.

Returns:: Group section-modified easting coordinates for each survey point.
Return type:: np.ndarray

get_survey_group_section_northing_mod()[source]#

Get group section-modified northing coordinates for each survey point.

Returns:: Group section-modified northing coordinates for each survey point.
Return type:: np.ndarray

class refpy.oos_tools.OOSDespiker(*, development=None, survey_type=None, pipeline_group=None, group_section_type=None, y, window=11, sigma=3.0)[source]#

Bases: object

This class identifies and removes outlier values from survey data using a rolling window sigma-clipping algorithm. Outliers are replaced with NaN, preserving the original data alignment and group structure.

Parameters:

development (array-like, optional) – Development identifier for each survey point. If not provided, a default value is used.
survey_type (array-like, optional) – Survey type for each survey point. If not provided, a default value is used.
pipeline_group (array-like, optional) – Pipeline group for each survey point. If not provided, a default value is used.
group_section_type (array-like, optional) – Group section type for each survey point. If not provided, a default value is used.
x (array-like) – Coordinates associated with the signal for each survey point. These can be either easting values or arc lengths, depending on context.
y (array-like) – Signal values for each survey point. These can be either northing values or curvatures, depending on context.
window (int, optional) – Size of the rolling window for sigma-clipping (default: 11).
sigma (float, optional) – Sigma threshold for outlier detection (default: 3.0).

get_y_despike_mean()[source]#

Get the despiked y values (i.e., northings or curvatures) for each survey point.

Returns:: Despiked y values for each survey point.
Return type:: np.ndarray

get_y_despike_nan()[source]#

Get the despiked y values (i.e., northings or curvatures) for each survey point.

Returns:: Despiked y values for each survey point.
Return type:: np.ndarray

class refpy.oos_tools.OOSCurvature(*, development=None, survey_type=None, pipeline_group=None, group_section_type=None, x, y)[source]#

Bases: object

Computes geometric curvature, arc length, and tangent angle for grouped pipeline survey data.

This class calculates the arc length, tangent angle, and curvature of the pipeline defined by easting and northing coordinates, for each group section. The results are aligned with the original data and are useful for geometric analysis and further signal processing.

Parameters:

development (array-like, optional) – Development identifier for each survey point. If not provided, a default value is used.
survey_type (array-like, optional) – Survey type for each survey point. If not provided, a default value is used.
pipeline_group (array-like, optional) – Pipeline group for each survey point. If not provided, a default value is used.
group_section_type (array-like, optional) – Group section type for each survey point. If not provided, a default value is used.
x (array-like) – These can be either eastings or anonymised eastings, depending on context.
y (array-like) – These can be either northings or anonymised northings, depending on context.

get_arc_length()[source]#

Get the arc length of the pipeline.

Returns:: Arc length for each survey point.
Return type:: np.ndarray

get_angle()[source]#

Get the tangent angle of the pipeline.

Returns:: Tangent angle for each survey point.
Return type:: np.ndarray

get_curvature()[source]#

Get the curvature of the pipeline.

Returns:: Curvature for each survey point.
Return type:: np.ndarray

class refpy.oos_tools.FFTSmoother(*, development=None, survey_type=None, pipeline_group=None, group_section_type=None, x, y, cutoff)[source]#

Bases: object

Class to perform FFT smoothing and spectral analysis on the pipeline survey data, grouped by (development, survey type, pipeline group, group section type).

Parameters:

development (array-like, optional) – Development identifier for each survey point. If not provided, a default value is used.
survey_type (array-like, optional) – Survey type for each survey point. If not provided, a default value is used.
pipeline_group (array-like, optional) – Pipeline group for each survey point. If not provided, a default value is used.
group_section_type (array-like, optional) – Group section type for each survey point. If not provided, a default value is used.
x (array_like, mandatory) – Coordinates associated with the signal for each survey point. These can be either anonymised easting values or arc lengths, depending on context.
y (array_like, mandatory) – Signal values for each survey point. These can be either anonymised northing values or curvatures, depending on context.
cutoff (float or None, mandatory) – Wavelength or curvature cutoff for FFT filtering. If None, no filtering is applied.

get_y_smooth()[source]#

Get the FFT smoothed y values.

Returns:: FFT smoothed y values for each survey point.
Return type:: np.ndarray

get_freqs()[source]#

Get the filtered frequency values.

Returns:: Filtered frequency values for each survey point.
Return type:: np.ndarray

get_fft()[source]#

Get the filtered FFT values.

Returns:: Filtered FFT values for each survey point.
Return type:: np.ndarray

get_freqs_raw()[source]#

Get the raw (unfiltered) frequency values.

Returns:: Raw frequency values for each survey point.
Return type:: np.ndarray

get_fft_raw()[source]#

Get the raw (unfiltered) FFT values.

Returns:: Raw FFT values for each survey point.
Return type:: np.ndarray

get_psd_development()[source]#

Get the PSD development values.

Returns:: PSD development values for each group.
Return type:: list

get_psd_survey_type()[source]#

Get the PSD survey type values.

Returns:: PSD survey type values for each group.
Return type:: list

get_psd_pipeline_group()[source]#

Get the PSD pipeline group values.

Returns:: PSD pipeline group values for each group.
Return type:: list

get_psd_group_section_type()[source]#

Get the PSD group section type values.

Returns:: PSD group section type values for each group.
Return type:: list

get_psd_freqs()[source]#

Get the PSD frequency arrays.

Returns:: PSD frequency arrays for each group.
Return type:: list

get_psd_vals()[source]#

Get the PSD values arrays.

Returns:: PSD values arrays for each group.
Return type:: list

get_x_recon()[source]#

Get the reconstructed x (easting) coordinates.

Returns:: Reconstructed x coordinates for each survey point.
Return type:: np.ndarray

get_y_recon()[source]#

Get the reconstructed y (northing) coordinates.

Returns:: Reconstructed y coordinates for each survey point.
Return type:: np.ndarray

class refpy.oos_tools.GaussianSmoother(*, development=None, survey_type=None, pipeline_group=None, group_section_type=None, x, y, bandwidth)[source]#

Bases: object

Class to perform Gaussian kernel smoothing on the pipeline survey, grouped by (development, survey type, pipeline group, group section type).

Parameters:

development (array-like, optional) – Development identifier for each survey point. If not provided, a default value is used.
survey_type (array-like, optional) – Survey type for each survey point. If not provided, a default value is used.
pipeline_group (array-like, optional) – Pipeline group for each survey point. If not provided, a default value is used.
group_section_type (array-like, optional) – Group section type for each survey point. If not provided, a default value is used.
x (array-like) – Anonymised eastings for each survey point.
y (array-like) – Anonymised northings for each survey point.
bandwidth (float) – Bandwidth for the Gaussian kernel.

get_y_smooth()[source]#

Get the smoothed y values (i.e., northings) for each survey point.

Returns:: Smoothed y values for each survey point.
Return type:: np.ndarray