Enhancing Visual Question Answering in Vietnamese Using Large Language Models Combined with OCR Systems
Tác giả
Tài liệu tham khảo
[1] Zakari, R.Y., et al.: VQA and visual reasoning: an overview of recent datasets, methods and challenges. arXiv:2212.13296 [cs.CV] (2022)
[2] Singh, A., et al.: Towards VQA models that can read. arXiv:1904.08920 [cs.CL] (2019)
[3] Agrawal, A., et al.: VQA: visual question answering. arXiv:1505.00468 [cs.CL] (2016)
[4] Touvron, H., et al.: LLaMA: open and efficient foundation language models. arXiv:2302.13971 [cs.CL] (2023)
[5] Radford, A., et al.: Learning transferable visual models from natural language supervision. arXiv:2103.00020 [cs.CV] (2021)
[6] Dai, W., et al.: InstructBLIP: towards general-purpose vision-language models with instruction tuning. arXiv:2305.06500 [cs.CV] (2023)
[7] Liu, H., et al.: Visual instruction tuning. arXiv:2304.08485 [cs.CV] (2023)
[8] Nguyen, N.H., et al.: OpenViVQA: task, dataset, and multimodal fusion models for visual question answering in Vietnamese. Inf. Fusion 100, 101868 (2023). https://doi.org/10.1016/j.inffus.2023.101868. ISSN 1566-2535
[9] Mishra, A., et al.: OCR-VQA: visual question answering by reading text in images. Sydney, Australia (2019). https://doi.org/10.1109/ICDAR.2019.00156. Accessed 15 July 2024
[10] Zhai, X., et al.: Sigmoid loss for language image pre-training. arXiv:2303.15343 [cs.CV] (2023)
[11] Baidu. PaddleOCR: Awesome multilingual OCR toolkits based on PaddlePaddle (2024). https://github.com/PaddlePaddle/PaddleOCR
[12] Cho, J., et al.: Unifying vision-and-language tasks via text generation. arXiv:2102.02779 [cs.CL] (2021)
[13] MiniCPM-V Team. MiniCPM-V 2.0: An Efficient End-side MLLM with Strong OCR and Understanding Capabilities (2024). https://openbmb.vercel.app/minicpm-v-2-en
[14] Papineni, K., et al.: BLEU: a method for automatic evaluation of machine translation. In: Isabelle, P., Charniak, E., Lin, D. (eds.) Philadelphia, Pennsylvania, USA (2002). https://doi.org/10.3115/1073083.1073135, https://aclanthology.org/P02-1040. Accessed 27 July 2024
[15] Vedantam, R., Zitnick, C.L., Parikh, D.: CIDEr: consensus-based image description evaluation. arXiv:1411.5726 [cs.CV] (2015)
[16] Dosovitskiy, A., et al.: An image is worth 16$$\times $$16 words: transformers for image recognition at scale. arXiv:2010.11929 [cs.CV] (2021)
[17] Xue, L., et al.: mT5: a massively multilingual pre-trained text-to-text transformer. arXiv:2010.11934 [cs.CL] (2021)
[18] Hu, E.J., et al.: LoRA: low-rank adaptation of large language models. arXiv:2106.09685 [cs.CL] (2021)