Table of Contents

Class DataLakeStorageService

Namespace
FoundationaLLM.Common.Services.Storage
Assembly
FoundationaLLM.Common.dll

Provides access to Azure Data Lake blob storage.

public class DataLakeStorageService : StorageServiceBase, IStorageService
Inheritance
DataLakeStorageService
Implements
Inherited Members
Extension Methods

Constructors

DataLakeStorageService(BlobStorageServiceSettings, ILogger<DataLakeStorageService>)

Initializes a new instance of the DataLakeStorageService with the specified options and logger.

public DataLakeStorageService(BlobStorageServiceSettings storageSettings, ILogger<DataLakeStorageService> logger)

Parameters

storageSettings BlobStorageServiceSettings

The BlobStorageServiceSettings object with the blob storage settings.

logger ILogger<DataLakeStorageService>

The logger used for logging.

DataLakeStorageService(IOptions<BlobStorageServiceSettings>, ILogger<DataLakeStorageService>)

Initializes a new instance of the DataLakeStorageService with the specified options and logger.

public DataLakeStorageService(IOptions<BlobStorageServiceSettings> options, ILogger<DataLakeStorageService> logger)

Parameters

options IOptions<BlobStorageServiceSettings>

The options object containing the BlobStorageServiceSettings object with the blob storage settings.

logger ILogger<DataLakeStorageService>

The logger used for logging.

Properties

StorageAccountName

The name of the storage account.

public string StorageAccountName { get; }

Property Value

string

Methods

CopyFileAsync(string, string, string, CancellationToken)

Copies a file from one location to another within the specified storage container.

public Task CopyFileAsync(string containerName, string sourceFilePath, string destinationFilePath, CancellationToken cancellationToken = default)

Parameters

containerName string

The name of the container.

sourceFilePath string

The path of the file to copy.

destinationFilePath string

The path of the destination.

cancellationToken CancellationToken

The cancellation token that signals that operations should be cancelled.

Returns

Task

CreateClientFromAccountKey(string, string)

Creates a storage client from an account name and an account key.

protected override void CreateClientFromAccountKey(string accountName, string accountKey)

Parameters

accountName string

The storage account name.

accountKey string

The storage account key.

CreateClientFromConnectionString(string)

Creates a storage client from a connection string.

protected override void CreateClientFromConnectionString(string connectionString)

Parameters

connectionString string

The storage connection string.

CreateClientFromIdentity(string)

Create a storage client from an account name using the default identity for authentication.

protected override void CreateClientFromIdentity(string accountName)

Parameters

accountName string

The storage account name.

DeleteFileAsync(string, string, CancellationToken)

Deletes a file from storage.

public Task DeleteFileAsync(string containerName, string filePath, CancellationToken cancellationToken = default)

Parameters

containerName string

The name of the container where the file is located.

filePath string

The path of the file to delete.

cancellationToken CancellationToken

The cancellation token that signals that operations should be cancelled.

Returns

Task

FileExistsAsync(string, string, CancellationToken)

Checks if a file exists on the storage.

public Task<bool> FileExistsAsync(string containerName, string filePath, CancellationToken cancellationToken = default)

Parameters

containerName string

The name of the container where the file is located.

filePath string

The path of the file to read.

cancellationToken CancellationToken

The cancellation token that signals that operations should be cancelled.

Returns

Task<bool>

GetFilePathsAsync(string, string?, bool, CancellationToken)

Retrieves a list file paths in the specific directory.

public Task<List<string>> GetFilePathsAsync(string containerName, string? directoryPath = null, bool recursive = true, CancellationToken cancellationToken = default)

Parameters

containerName string

Name of the container, file system or Workspace name.

directoryPath string

Directory to list file contents

recursive bool

Recurse through child folders

cancellationToken CancellationToken

Determines if a request should be cancelled.

Returns

Task<List<string>>

List of individual file paths located in the container.

GetMatchingFilePathsAsync(string, string, CancellationToken)

Retrieves a list of file paths that match a specific pattern.

public Task<List<string>> GetMatchingFilePathsAsync(string containerName, string filePathPattern, CancellationToken cancellationToken = default)

Parameters

containerName string

The name of the container.

filePathPattern string

The file name pattern to match.

cancellationToken CancellationToken

The cancellation token that signals that operations should be cancelled.

Returns

Task<List<string>>

ReadFile(string, string)

Reads the binary content of a specified file from the storage.

public BinaryData ReadFile(string containerName, string filePath)

Parameters

containerName string

The name of the container where the file is located.

filePath string

The path of the file to read.

Returns

BinaryData

The binary content of the file.

ReadFileAsync(string, string, CancellationToken)

Reads the binary content of a specified file from the storage.

public Task<BinaryData> ReadFileAsync(string containerName, string filePath, CancellationToken cancellationToken = default)

Parameters

containerName string

The name of the container where the file is located.

filePath string

The path of the file to read.

cancellationToken CancellationToken

The cancellation token that signals that operations should be cancelled.

Returns

Task<BinaryData>

The binary content of the file.

UpdateJSONFileAsync<T>(string, string, Func<T, T>, CancellationToken)

Updates a specified JSON file in the storage.

public Task UpdateJSONFileAsync<T>(string containerName, string filePath, Func<T, T> objectTransformer, CancellationToken cancellationToken) where T : class

Parameters

containerName string

The name of the container where the file is located.

filePath string

The path of the file to update.

objectTransformer Func<T, T>

A function that updates the object persisted in the JSON file.

cancellationToken CancellationToken

The cancellation token that signals that operations should be cancelled.

Returns

Task

Type Parameters

T

The type of the object persisted in the JSON file.

WriteFileAsync(string, string, Stream, string?, CancellationToken)

Writes the binary content to a specified file from the storage.

public Task WriteFileAsync(string containerName, string filePath, Stream fileContent, string? contentType, CancellationToken cancellationToken)

Parameters

containerName string

The name of the container where the file is located.

filePath string

The path of the file to read.

fileContent Stream

The binary content written to the file.

contentType string

An optional content type.

cancellationToken CancellationToken

The cancellation token that signals that operations should be cancelled.

Returns

Task

WriteFileAsync(string, string, string, string?, CancellationToken)

Writes the string content to a specified file from the storage.

public Task WriteFileAsync(string containerName, string filePath, string fileContent, string? contentType, CancellationToken cancellationToken)

Parameters

containerName string

The name of the container where the file is located.

filePath string

The path of the file to read.

fileContent string

The string content written to the file.

contentType string

An optional content type.

cancellationToken CancellationToken

The cancellation token that signals that operations should be cancelled.

Returns

Task
Edit this page