Function chunkDirectoryStructure

  • chunkDirectoryStructure(structure, delimiter?, maxTokensPerChunk?, charsPerToken?): ChunkedDirectoryNode

  • Chunks all files in a directory structure tree. Returns a new tree with chunks added to all file nodes (immutable).

    Parameters

    • structure: DirectoryNode

      Directory structure from getDirectoryStructure()

    • delimiter: RegExp = ...

      Regex pattern to split files (default: 2+ newlines). Must have 'g' flag.

    • OptionalmaxTokensPerChunk: number

      Optional maximum tokens per chunk (will split if exceeded)

    • charsPerToken: number = 2.5

      Characters per token estimate (default: 2.5 for code)

    Returns ChunkedDirectoryNode

    New hierarchical structure with chunks added to file nodes

    Example

    const structure = getDirectoryStructure('/path/to/dir');
    const chunked = chunkDirectoryStructure(structure);

    Example

    // With size limits for embedding
    const chunked = chunkDirectoryStructure(structure, /\n{3,}/g, 6000, 2.5);

Settings

Member Visibility