Let ρ be a two-dimensional modulo p representation of the absolute Galois group of a totally real number field. Under the assumptions that ρ has a large image and admits a low-weight crystalline modular deformation we show that any low-weight crystalline deformation of ρ unramified outside a finite set of primes will be modular. We follow the approach of Wiles as generalized by Fujiwara. The main new ingredient is an Ihara-type lemma for the local component at ρ of the middle degree cohomology of a Hilbert modular variety. As an application we relate the algebraic p-part of the value at one of the adjoint L-function associated with a Hilbert modular newform to the cardinality of the corresponding Selmer group.