以前のポスト 兼好法師に質問する を MCP サーバにしてみようと思ったのだが、 MCPサーバの実装は Spring AI を使いたい。 そこで、 Python で実装している兼好法師が質問にこたえる部分を Spring AI で実装することにした。 その覚え書きです。 ただし、このポストではまだテキストを埋め込み(Embeddings)に変換できることを確認しただけです。
Spring AI でテキストの embeddings を計算する方法はここ https://docs.spring.io/spring-ai/reference/api/embeddings/onnx.html です。
Spring AI で 埋め込みを計算するためにはモデルが必要ですが、 （ここでは intfloat/multilingual-e5-small を使います） そのモデルをまずは ONNX 形式に変換する必要があります。
To run things in Java, we need to serialize the Tokenizer and the Transformer Model into ONNX format.
まず venv 環境をつくって、必要なライブラリを入れます:
現在の Python のバージョンを確認:
$ python3 --version Python 3.9.6
$ python3 -m venv venv
$ source ./venv/bin/activate
$ (venv) pip install --upgrade pip
$ (venv) pip install optimum onnx onnxruntime sentence-transformers
sentence-transformers は必須ではなかった。詳細後述する。
optimum-cli コマンドを使って変換を実行:
$ (venv) optimum-cli export onnx --model intfloat/multilingual-e5-small onnx
モデルを intfloat/multilingual-e5-small を指定します。 onnx は変換した結果のエキスポート先ディレクトリの指定です。
ちなみに small ではなく base を使いたければ次のようにモデル名(intfloat/multilingual-e5-base)を指定すればよい:
$ (venv) optimum-cli export onnx --model intfloat/multilingual-e5-base onnx
./onnx 以下に次のファイルが生成されました:
.
└── onnx
├── config.json
├── model.onnx
├── special_tokens_map.json
├── tokenizer.json
└── tokenizer_config.json
それではテキストの埋め込を計算するコードを書く:
// ai.main.kts
@file:Repository("https://repo1.maven.org/maven2/")
@file:DependsOn("org.springframework.ai:spring-ai-transformers:1.1.0-M1")
import org.springframework.ai.transformers.TransformersEmbeddingModel
val path="./onnx"
val embeddingModel = TransformersEmbeddingModel()
embeddingModel.setTokenizerResource("${path}/tokenizer.json")
embeddingModel.setModelResource("${path}/model.onnx")
embeddingModel.setResourceCacheDirectory("./tmp_onnx")
embeddingModel.setTokenizerOptions( mapOf("padding" to "true") )
embeddingModel.afterPropertiesSet()
val embeddings = embeddingModel.embed(listOf("Hello world", "World is big"))
println( embeddings )
実行:
$ kotlin ai.main.kts
java.lang.OutOfMemoryError: Java heap space
メモリが足りない。ヒープを増やすオプションをつけて再度実行:
$ kotlin -J-Xmx4g -J-Xms1g ai.main.kts
今度は...
java.lang.IllegalArgumentException: The generative output names don't contain expected: last_hidden_state.
Consider one of the available model outputs: token_embeddings, sentence_embedding
token_embeddings, sentence_embedding のどちらか一つを考慮して、といわれたので...
// embeddingModel.setModelOutputName("sentence_embedding")
embeddingModel.setModelOutputName("token_embeddings")
sentence_embedding の方を指定すると次のエラーが出た:
java.lang.ClassCastException: class [[F cannot be cast to class [[[F ([[F and [[[F are in module java.base of loader 'bootstrap')token_embeddings を指定したらエラーなしで作動したので、とりあえずこれで。
公式の説明にも記載あった: https://docs.spring.io/spring-ai/reference/api/embeddings/onnx.html
If you get an error like The generative output names don’t contain expected: last_hidden_state. Consider one of the available model outputs: token_embeddings,
you need to set the model output name to a correct value per your models. Consider the names listed in the error message.
これを足す:
embeddingModel.setModelOutputName("token_embeddings")
完成したコード:
// ai.main.kts
@file:Repository("https://repo1.maven.org/maven2/")
@file:DependsOn("org.springframework.ai:spring-ai-transformers:1.1.0-M1")
import org.springframework.ai.transformers.TransformersEmbeddingModel
val path="./onnx"
val embeddingModel = TransformersEmbeddingModel()
embeddingModel.setTokenizerResource("${path}/tokenizer.json")
embeddingModel.setModelResource("${path}/model.onnx")
embeddingModel.setResourceCacheDirectory("./tmp_onnx")
embeddingModel.setTokenizerOptions( mapOf("padding" to "true") )
//embeddingModel.setModelOutputName("sentence_embedding")
embeddingModel.setModelOutputName("token_embeddings")
embeddingModel.afterPropertiesSet()
val embeddings = embeddingModel.embed(listOf("Hello world", "World is big"))
println( embeddings )
実行:
$ kotlin -J-Xmx4g -J-Xms1g ai.main.kts
[[F@7e5c04a4, [F@6212dc7d]
できた。 ただ生成された埋め込みの様子がよくわからん!
ai.main.kts に以下のコードを足す:
val r: FloatArray = embeddings[0]
println( r.size )
val t = r.map { it } .joinToString(",")
println( t )
実行:
$ kotlin -J-Xmx4g -J-Xms1g ai.main.kts
384
0.18695532,0.12300663,-0.20084214,-0.3653491,0.41024983,-0.14747803,0.08159739,0.29558343,0.15827098,0.012738429,0.26965857,0.10375314,0.28951907,-0.21678391,-0.26570982,0.13628082,0.3507977,-0.09281093,-0.09947128,-0.18554638,0.24574625,0.023116283,-0.26029286,0.19993949,0.1713526,0.31221688,-0.16121025,0.10319413,0.16245577,-0.19532895,-0.1656973,0.017596882,0.20146602,-0.101997316,0.29879385,0.53949827,-0.15860699,-0.23886988,0.3735332,-0.24628216,-0.23313916,0.26397172,-0.04851482,0.27509582,0.19380784,0.36544657,-0.13917606,0.18020459,-0.33823967,0.017737992,-0.3174128,0.10421085,0.1104294,0.3214314,0.13241178,-0.48667046,-0.1402325,-0.33841604,-0.38276964,0.22176653,0.2760676,0.07363861,0.005911682,0.1848644,0.2115907,0.17016104,0.15826268,0.009890644,-0.22557944,-0.014399752,-0.13819945,0.19335589,-0.0043554185,-0.25046772,0.14603025,0.028778132,-0.24427061,-0.29373258,0.27434343,-0.2546351,-0.37315565,-0.372578,-0.27005148,0.22948886,-0.21626684,0.28107285,0.117786705,-0.18297708,0.24314825,-0.14060545,0.32913494,-0.031963278,-0.2500031,-0.19760428,-0.17815799,-0.2521238,-0.25949913,0.30976036,0.2698413,-0.248139,0.35484132,-0.07809754,0.19677079,-0.19244397,-0.36154538,0.043155078,0.27528554,-0.26513883,0.236152,-0.19678685,-0.10685483,0.1275737,0.24269193,0.28680193,-0.43602633,-0.3034871,-0.11849543,-0.08327028,0.115385935,-0.27078158,0.23747239,-0.22608277,-0.17298377,-0.16045505,-0.2473146,-0.40074384,-0.042779345,0.06864669,0.0014373418,0.18400726,0.31365815,0.16797186,0.24942179,0.2969018,0.11464268,0.11007747,-0.07212605,0.014100003,-0.17725462,-0.12056914,-0.14645533,0.26170075,-0.078255884,0.3678364,0.14758016,0.04345471,0.43687004,-0.1144252,0.36756635,-0.14615802,0.16285989,-0.04162369,0.19458792,0.07573651,0.15634125,-0.14078537,-0.22636274,-0.21765247,0.18365628,0.37174982,-0.2609347,-0.333062,-0.32835317,-0.14779517,-0.21346954,-0.13232456,0.07993832,0.26463813,-0.12537289,-0.20399268,-0.2758444,0.24411434,-0.41284943,0.17052479,-1.3065245E-4,0.19126026,-0.20889065,0.2827824,0.26731855,0.118232794,-0.1051316,-0.054943215,-0.4615339,-0.055434175,-0.37978014,-0.17284837,-0.20443323,0.19484548,0.11338342,-0.23445857,-0.19150884,0.04881409,-0.2256137,-0.33780316,-0.2495761,0.22053917,-0.33134538,0.21256682,0.23629794,0.23214051,-0.30253303,-0.35781783,0.170206,0.18427426,0.27356327,0.17921549,-0.23299733,0.26739442,-0.27565733,0.11412121,0.34121296,-0.2868365,-0.39162445,0.2833599,-0.42700684,-0.21444976,-0.009102731,0.35790366,-0.2202142,0.15563948,0.3381283,-0.06610117,0.14918713,-0.15787144,-0.30779943,0.06400008,0.1220378,-0.34007746,-0.1984351,0.24265295,-0.31923813,-0.14670417,-0.42122662,-0.3199933,-0.12770376,-0.28555125,-0.025857331,0.10735807,0.18567069,-0.1219562,-0.022736467,-0.16909854,0.29976672,-0.35374722,0.21822345,-0.08486513,-0.13081446,0.21305595,-0.015793318,0.130595,0.08211035,-0.27186286,-0.17413476,-0.32550544,-0.27864385,0.06580472,0.33567375,0.24089174,-0.35705078,0.09422729,0.18161923,-0.2543709,0.13206488,0.19318222,0.27368408,-0.08580873,-0.17278238,-0.055559963,-0.31210902,-0.26314846,-0.2801297,0.044377252,0.22743899,-0.1704622,-0.08318159,-0.19642054,0.41020012,0.35509008,-0.1528273,-0.14533107,0.14362177,0.1850425,0.31265548,0.23835218,0.31088933,-0.05899048,-0.1530506,0.22414194,-0.2709431,-0.16596824,0.15667585,-0.22643222,0.11992252,-0.17286323,0.29802123,0.07310705,0.12166649,0.21364151,-0.26774558,0.45415914,-0.1470938,-0.20883238,0.24577084,0.11639095,-0.18179011,0.29139987,0.055752896,-0.06883755,0.19550708,0.15378194,0.19932817,0.2822699,-0.22952028,-0.28358495,0.342924,0.37734506,-0.11424032,-0.088431574,-0.0067914203,-0.19940494,-0.22047886,-0.17767939,-0.13228649,-0.10238283,0.12273155,0.15378709,-0.4242657,-0.1461662,0.3548926,0.009869736,0.12027974,-0.23005396,-0.18816659,0.32363814,-0.19420955,-0.119386226,-0.07983379,0.20986739,-0.3055876,-0.26521766,0.0025524208,0.07181097,-0.35338742,0.15752621,-0.18240607,-0.28903025,0.4261836,-0.22137806,-0.31716,0.3432212,0.07653692,-0.2555412,0.2436378,0.3659403,-0.08216821,0.23399141,-0.16614348,-0.046037853,-0.0062045977,0.052126825,-0.14807597,-0.16615559,0.29594448,0.25213003,0.31950134,0.31734812,-0.12638558,-0.114243925,0.28132203,-0.18061683,0.04457979,0.23281153,-0.28741938,-0.08258661,-0.08318468,-0.18009889,-0.2043373,0.1594628,-0.115977414,-0.26773915,0.12501805,0.42874938,0.22647244,0.08447567
384個の値をもつ FloatArray (ベクター）情報＝埋め込み、に変換されていた。
ライブラリを入れる部分で...
これを:
$ (venv) pip install optimum onnx onnxruntime sentence-transformers
これにする:
$ (venv) pip install optimum onnx onnxruntime
全体ではこうなる:
$ python3 -m venv venv
$ source ./venv/bin/activate
$ (venv) pip install --upgrade pip
$ (venv) pip install optimum onnx onnxruntime
multilingual-e5-small を ONNX モデルに変換する:
$ (venv) optimum-cli export onnx --model intfloat/multilingual-e5-small onnx
このとき次のメッセージが出た:
The library name was inferred as sentence_transformers, which is not installed.
Falling back to transformers to avoid breaking the export.
sentence_transformers がインストールされてないから transformers にフォールバックすると。 いったい、 sentence_transformers と transformers と何が違うのであろうか？
claude に尋ねてみた:
The key difference is that sentence_transformers is a specialized library built on top of transformers for creating sentence and text embeddings, while transformers is the broader, more general library for working with transformer models.
ということは文章（テキスト）の埋め込みを得たいのであれば、やはり sentence_transformers を使う必要があるということか。
コード:
// ai.main.kts
@file:Repository("https://repo1.maven.org/maven2/")
@file:DependsOn("org.springframework.ai:spring-ai-transformers:1.1.0-M1")
import org.springframework.ai.transformers.TransformersEmbeddingModel
val path="./onnx"
val embeddingModel = TransformersEmbeddingModel()
embeddingModel.setTokenizerResource("${path}/tokenizer.json")
embeddingModel.setModelResource("${path}/model.onnx")
embeddingModel.setResourceCacheDirectory("./tmp_onnx")
embeddingModel.setTokenizerOptions( mapOf("padding" to "true") )
//embeddingModel.setModelOutputName("sentence_embedding")
//embeddingModel.setModelOutputName("token_embeddings")
embeddingModel.afterPropertiesSet()
val embeddings = embeddingModel.embed(listOf("Hello world", "World is big"))
println( embeddings )
val r: FloatArray = embeddings[0]
println( r.size )
val t = r.map { it } .joinToString(",")
println( t )
問題の部分はコメントアウトしてある:
//embeddingModel.setModelOutputName("sentence_embedding") //embeddingModel.setModelOutputName("token_embeddings")
これで実行:
$ kotlin -J-Xmx4g -J-Xms1g ai.main.kts
[[F@116c43d4, [F@b6a5cf5]
384
0.18695532,0.123006... 省略