KYHTQT.

Enhancing Visual Question Answering in Vietnamese Using Large Language Models Combined with OCR Systems

Năm XB 2024 Tạp chí / Hội thảo Lecture Notes in Networks and Systems Volume 1205 LNNS Đơn vị NT&TT DOI / Link https://doi.org/10.1007/978-3-031-80943-9_34 ↗

Tác giả

Tài liệu tham khảo

[1] Zakari, R.Y., et al.: VQA and visual reasoning: an overview of recent datasets, methods and challenges. arXiv:2212.13296 [cs.CV] (2022)

[2] Singh, A., et al.: Towards VQA models that can read. arXiv:1904.08920 [cs.CL] (2019)

[3] Agrawal, A., et al.: VQA: visual question answering. arXiv:1505.00468 [cs.CL] (2016)

[4] Touvron, H., et al.: LLaMA: open and efficient foundation language models. arXiv:2302.13971 [cs.CL] (2023)

[5] Radford, A., et al.: Learning transferable visual models from natural language supervision. arXiv:2103.00020 [cs.CV] (2021)

[6] Dai, W., et al.: InstructBLIP: towards general-purpose vision-language models with instruction tuning. arXiv:2305.06500 [cs.CV] (2023)

[7] Liu, H., et al.: Visual instruction tuning. arXiv:2304.08485 [cs.CV] (2023)

[8] Nguyen, N.H., et al.: OpenViVQA: task, dataset, and multimodal fusion models for visual question answering in Vietnamese. Inf. Fusion 100, 101868 (2023). https://doi.org/10.1016/j.inffus.2023.101868. ISSN 1566-2535

[9] Mishra, A., et al.: OCR-VQA: visual question answering by reading text in images. Sydney, Australia (2019). https://doi.org/10.1109/ICDAR.2019.00156. Accessed 15 July 2024

[10] Zhai, X., et al.: Sigmoid loss for language image pre-training. arXiv:2303.15343 [cs.CV] (2023)

[11] Baidu. PaddleOCR: Awesome multilingual OCR toolkits based on PaddlePaddle (2024). https://github.com/PaddlePaddle/PaddleOCR

[12] Cho, J., et al.: Unifying vision-and-language tasks via text generation. arXiv:2102.02779 [cs.CL] (2021)

[13] MiniCPM-V Team. MiniCPM-V 2.0: An Efficient End-side MLLM with Strong OCR and Understanding Capabilities (2024). https://openbmb.vercel.app/minicpm-v-2-en

[14] Papineni, K., et al.: BLEU: a method for automatic evaluation of machine translation. In: Isabelle, P., Charniak, E., Lin, D. (eds.) Philadelphia, Pennsylvania, USA (2002). https://doi.org/10.3115/1073083.1073135, https://aclanthology.org/P02-1040. Accessed 27 July 2024

[15] Vedantam, R., Zitnick, C.L., Parikh, D.: CIDEr: consensus-based image description evaluation. arXiv:1411.5726 [cs.CV] (2015)

[16] Dosovitskiy, A., et al.: An image is worth 16$$\times $$16 words: transformers for image recognition at scale. arXiv:2010.11929 [cs.CV] (2021)

[17] Xue, L., et al.: mT5: a massively multilingual pre-trained text-to-text transformer. arXiv:2010.11934 [cs.CL] (2021)

[18] Hu, E.J., et al.: LoRA: low-rank adaptation of large language models. arXiv:2106.09685 [cs.CL] (2021)

Ghi chú

ICTA