Recent observations of neutron stars with gravitational waves and X-ray timing provide unprecedented access to the equation of state ( EoS ) of cold dense matter at densities difficult to realize in terrestrial experiments . At the same time , predictions for the EoS equipped with reliable uncertainty estimates from chiral effective field theory ( \chi \text { EFT } ) allow us to bound our theoretical ignorance . In this work , we analyze astrophysical data using a nonparametric representation of the neutron-star EoS conditioned on \chi \text { EFT } to directly constrain the underlying physical properties of the compact objects without introducing modeling systematics . We discuss how the data alone constrain the EoS at high densities when we condition on \chi \text { EFT } at low densities . We also demonstrate how to exploit astrophysical data to directly test the predictions of \chi \text { EFT } for the EoS up to twice nuclear saturation density , in order to estimate the density at which these predictions might break down . We find that the existence of massive pulsars , gravitational waves from GW170817 , and NICER observations of PSR J0030+0451 favor \chi \text { EFT } predictions for the EoS up to nuclear saturation density over a more agnostic analysis by as much as a factor of 7 for the quantum Monte Carlo ( QMC ) calculations used in this work . While \chi \text { EFT } predictions using QMC are fully consistent with gravitational-wave data up to twice nuclear saturation density , NICER observations suggest that the EoS stiffens relative to these predictions at or slightly above nuclear saturation density . Additionally , for these QMC calculations , we marginalize over the uncertainty in the density at which \chi \text { EFT } begins to break down , constraining the radius of a 1.4 M _ { \odot } neutron star to R _ { 1.4 } = 11.40 ^ { +1.38 } _ { -1.04 } ( 12.54 ^ { +0.71 } _ { -0.63 } ) km and the pressure at twice nuclear saturation density to p ( 2 n _ { \mathrm { sat } } ) = 14.2 ^ { +18.1 } _ { -8.4 } ( 28.7 ^ { +15.3 } _ { -15.0 } ) \mathrm { MeV } / \mathrm { fm } ^ { 3 } with massive pulsar and gravitational-wave ( and NICER ) data .