He added: "I am particularly concerned that this incident, which could have been much more serious, was able to happen in the first place."
The child was treated by paramedics at the Needles Battery scene at about 1430 BST before being taken to St Mary's Hospital in Newport.
Witnesses have been urged to contact police.