A 13B-parameter Japanese-optimized large language model developed by NII. Fine-tuned using LoRA on the Dolly and OASST datasets for improved instruction following.
Parameters
Undisclosed
Context Window
License
Proprietary
Release Date
2023-10-18
API Pricing
API pricing for this model is not yet available
Strengths
- •Optimized for Japanese language based on the llm-jp model foundation.
- •13B parameter size offers a good balance between capability and resource use.
- •Fine-tuned on popular instruction datasets (Dolly/OASST) via efficient LoRA.
Weaknesses
- •Performance is heavily dependent on the quality of the base llm-jp model.
- •LoRA fine-tuning might not capture as much instruction-following nuance as full SFT.
- •May have limitations in tasks outside the scope of its fine-tuning data.
Use Cases
- •Japanese chatbots and virtual assistants that follow instructions.
- •Tasks benefiting from instruction-tuning, like Q&A and task completion.
- •Research on applying LoRA fine-tuning to Japanese language models.