Dataset related classes

Dataset related classes

Functions

Properties

GArrowFileSystem * file-system Read / Write / Construct Only
GADatasetFileFormat * format Read / Write / Construct Only
GADatasetPartitioning * partitioning Read / Write / Construct Only
char * base-dir Read / Write
char * base-name-template Read / Write
GArrowFileSystem * file-system Read / Write
GADatasetFileWriteOptions * file-write-options Read / Write
guint max-partitions Read / Write
GADatasetPartitioning * partitioning Read / Write

Types and Values

Object Hierarchy

    GObject
    ├── GADatasetDataset
       ╰── GADatasetFileSystemDataset
    ╰── GADatasetFileSystemDatasetWriteOptions

Includes

#include <arrow-dataset-glib/arrow-dataset-glib.h>

Description

GADatasetDataset is a base class for datasets.

GADatasetFileSystemDataset is a class for file system dataset.

GADatasetFileSystemDatasetWriteOptions is a class for options to write a dataset to file system dataset.

Functions

gadataset_dataset_begin_scan ()

GADatasetScannerBuilder *
gadataset_dataset_begin_scan (GADatasetDataset *dataset,
                              GError **error);

Parameters

dataset

A GADatasetDataset.

 

error

Return location for a GError or NULL.

[nullable]

Returns

A newly created GADatasetScannerBuilder on success, NULL on error.

[transfer full][nullable]

Since: 5.0.0

gadataset_dataset_to_table ()

GArrowTable *
gadataset_dataset_to_table (GADatasetDataset *dataset,
                            GError **error);

Parameters

dataset

A GADatasetDataset.

 

error

Return location for a GError or NULL.

[nullable]

Returns

A loaded GArrowTable on success, NULL on error.

[transfer full][nullable]

Since: 5.0.0

gadataset_dataset_get_type_name ()

gchar *
gadataset_dataset_get_type_name (GADatasetDataset *dataset);

Parameters

dataset

A GADatasetDataset.

 

Returns

The type name of dataset .

It should be freed with g_free() when no longer needed.

Since: 5.0.0

gadataset_file_system_dataset_write_options_new ()

GADatasetFileSystemDatasetWriteOptions *
gadataset_file_system_dataset_write_options_new
                               (void);

Returns

The newly created GADatasetFileSystemDatasetWriteOptions.

Since: 6.0.0

gadataset_file_system_dataset_write_scanner ()

gboolean
gadataset_file_system_dataset_write_scanner
                               (GADatasetScanner *scanner,
                                GADatasetFileSystemDatasetWriteOptions *options,
                                GError **error);

Parameters

scanner

A GADatasetScanner that produces data to be written.

 

options

A GADatasetFileSystemDatasetWriteOptions.

 

error

Return location for a GError or NULL.

[nullable]

Returns

TRUE on success, FALSE on error.

Since: 6.0.0

Types and Values

GADATASET_TYPE_FILE_SYSTEM_DATASET_WRITE_OPTIONS

#define             GADATASET_TYPE_FILE_SYSTEM_DATASET_WRITE_OPTIONS

struct GADatasetFileSystemDatasetWriteOptionsClass

struct GADatasetFileSystemDatasetWriteOptionsClass {
  GObjectClass parent_class;
};

GADATASET_TYPE_FILE_SYSTEM_DATASET

#define             GADATASET_TYPE_FILE_SYSTEM_DATASET

struct GADatasetFileSystemDatasetClass

struct GADatasetFileSystemDatasetClass {
  GADatasetDatasetClass parent_class;
};

GADatasetFileSystemDataset

typedef struct _GADatasetFileSystemDataset GADatasetFileSystemDataset;

GADatasetFileSystemDatasetWriteOptions

typedef struct _GADatasetFileSystemDatasetWriteOptions GADatasetFileSystemDatasetWriteOptions;

Property Details

The “file-system” property

  “file-system”              GArrowFileSystem *

File system of the dataset.

Owner: GADatasetFileSystemDataset

Flags: Read / Write / Construct Only

Since: 5.0.0

The “format” property

  “format”                   GADatasetFileFormat *

Format of the dataset.

Owner: GADatasetFileSystemDataset

Flags: Read / Write / Construct Only

Since: 5.0.0

The “partitioning” property

  “partitioning”             GADatasetPartitioning *

Partitioning of the dataset.

Owner: GADatasetFileSystemDataset

Flags: Read / Write / Construct Only

Since: 6.0.0

The “base-dir” property

  “base-dir”                 char *

Root directory into which the dataset will be written.

Owner: GADatasetFileSystemDatasetWriteOptions

Flags: Read / Write

Default value: NULL

The “base-name-template” property

  “base-name-template”       char *

Template string used to generate fragment base names. {i} will be replaced by an auto incremented integer.

Owner: GADatasetFileSystemDatasetWriteOptions

Flags: Read / Write

Default value: NULL

Since: 6.0.0

The “file-system” property

  “file-system”              GArrowFileSystem *

GArrowFileSystem into which a dataset will be written.

Owner: GADatasetFileSystemDatasetWriteOptions

Flags: Read / Write

The “file-write-options” property

  “file-write-options”       GADatasetFileWriteOptions *

Options for individual fragment writing.

Owner: GADatasetFileSystemDatasetWriteOptions

Flags: Read / Write

The “max-partitions” property

  “max-partitions”           guint

Maximum number of partitions any batch may be written into.

Owner: GADatasetFileSystemDatasetWriteOptions

Flags: Read / Write

Allowed values: <= G_MAXINT

Default value: 1024

Since: 6.0.0

The “partitioning” property

  “partitioning”             GADatasetPartitioning *

GADatasetPartitioning used to generate fragment paths.

Owner: GADatasetFileSystemDatasetWriteOptions

Flags: Read / Write

Since: 6.0.0