The public administration domain plays a crucial role in contemporary governance systems, directly affecting the interaction between citizens and government agencies. With the rapid development of information technology, the digitization and automated processing of public administrative documents have become urgent requirements. This work builds a question-answer dataset for Vietnamese public administrative documents and evaluates an extractive question-answering model using PhoBERT. The reported experiment reaches an EM/F1 score of 40.72/57.33 for answer extraction.