When can I put my baby in a high chair ?

I have a 4 and a half month baby boy ... weighs 13 lbs and he tries to come forward when I place him on his bouncer. Wondering if I can sit him on a high chair. He holds his head up but it shakes a little when unsupported. Is it time to place him on a high chair ? Also can I feed him while he is sitting on his high chair at this age ? He has started on applesauce and pureed carrots ( 1st stage gerber foods ).