Composite materials can exhibit brittle fractures under certain conditions, which depend on their composition, structure, and loading environment. Nonlinear numerical modeling of crack propagation and fracture behavior in these materials is one of the challenging topics in computational mechanics. It often requires the creation of detailed finite element models to accurately represent the material's damage, which can lead to high computational costs. In this contribution, we illustrate how the Virtual Element Method (VEM), a recent numerical technique already employed to model complex microstructural behavior, can be combined with an Adaptive Mesh Refinement (AMR) procedure and a non-local Continuous Damage Mechanics (CDM) approach. This combination allows for efficient modeling of crack propagation in composite materials while maintaining a desired level of accuracy and minimizing computational resources. We provide and discuss several numerical examples to illustrate the performance of the proposed VEM-based mesh-adaptive technique, focusing on its accuracy, reliability in predicting crack propagation, and computational efficiency. Additionally, we examine the effects of parameters associated with the mesh-refining procedure.