gpt3_tokenizer

OpenAI GPT-3 Tokenizer

libo
Natural Language Processing (NLP)

Package Resources

gpt3_tokenizer
LiboShen/gpt3-tokenizer-elixir
Documentation
No Changelog

Stars

29

Unreleased Activity

Up to date

First Release

2

years ago

Last Release

1

years ago

Downloads

18.3k

last 90 days

Last Stable Version

0.1.1

Activity

last year

Pull Requests

Open 0
Merged 0

Latest Merged Pull Requests

No Github activity

Issues

Open 0
Closed 0

Version 0.1.1

Published Jan 2024 •
Elixir Version Requirement ~> 1.0
Documentation for 0.1.1

Dependencies

2
jason ~> 1.2

A blazing fast JSON parser and generator in pure Elixir.

memoize ~> 1.4

A method caching macro for elixir using CAS on ETS

No changelog for 0.1.1
Inspect source for 0.1.1

Related Packages

22 packages

inflex

2.1.0

An Elixir library for handling word inflections.

tokenizers

0.5.1

Bindings to Hugging Face Tokenizers for Elixir

text_chunker

0.4.0

An Elixir library for semantic text chunking.

fast_ngram

1.2.0

A fast and unicode aware letter & word N-gram library written in Elixir.