Take a big sigh of relief, parents. Outfitting our littlest
reenactors is a snap compared to adults and older children. Styles for
infants were relatively standardized and simple.
With the exception of christening gowns, clothing generally was unadorned with lace & finery, as they were designed to be washed with frequency. Hard working moms had enough work to do raising children and running the household without adding hours of additional washing and ironing.
Until children were potty trained, both boys & girls were dressed the same. They wore dresses or frocks of white cotton (easy to wash & bleach) or natural fabrics in neutral tones. Baby gowns were often long, with the hemline well below the feet. This could be to keep their little toes warm and covered, or to make crawling difficult so baby would stay put and out of danger. Babies of poor families might be dressed only in an older sibling's hand-me-down shirt.
Be sure to have a bonnet for baby to wear when outside. Most period baby bonnets resembled women's sunbonnets. Saque sweaters, bonnets and booties knitted or crocheted in natural fibers (neutral colors, please!) would be appropriate.
Cotton diapers were a standard. In lieu of rubber pants, babies wore "Soakers", pants knitted or crocheted of untreated/unwashed wool. The lanolin in unwashed wool was naturally water repellant. If you must use disposable diapers or plastic pants, PLEASE cover them with a cotton diaper or homemade pants of period fabrics.
When
babies began to crawl and walk, they were dressed in shorter dresses.
They wore little petticoats under them, and drawers as soon as they were out
of diapers. Boys and Girls were dressed identically. (For more detailed
information on this practice, refer to the School
of the Civilian Children's Clothing Site, a very good article)
Toddlers wore booties or boots, and stockings. In cooler weather stockings
or leggings are essential for warmth. Some parents use tights because
they stay up better. If you use tights, get some in a plain weave, then
put their soakers or pants over them to camouflage the modern seaming on tights
Although Civil War era boys & girls would be dressed in a style we consider feminine today, it was a common practice of the times. This can be a little tough for some of our young gentlemen, who already have a few ideas about how they want to look. If the period style gives little Bobby fits, then dress him in a little tunic shirt and short pants. Although usually only older children wore this type of clothing, Bobby will look fine and feel better about himself, and peace shall be returned to the household.
A type of garment called "Button Suits" were popular for young boys. This garment was composed of short or long pants and a suit type of jacket that buttoned to the pants at the waistband. This kept their pants up, shirts down and gave them a very neat appearance. An undershirt or shirt is worn under the button suit.
Girl's hair was parted in the middle. Boys hair was parted on the side or swirled into a little tuft on top. This was often the only way to tell the sex of an infant in a photo, since boys & girls were dressed the same.
Keep in mind the economic status of your reenactment character. Although frilly dresses in the latest styles would adorn both boys and girls of wealthy families, poor families would dress their children quite differently. Poor or frontier families would likely dress their little tykes in a simple shift or hand-me-down clothing altered to their size.
Modern baby bottles bear little resemblance to period nursers, but they can be disguised by using bottle cozies. These are little fabric sacks that fit over the bottle and are held in place by a casing of elastic just beneath the nipple. Pacifiers of today little resemble their 1863 counterparts, do the best you can and shun bright colored plastics. White would be best, pretend it is ivory...
Period correct toys would be made of wood or natural fabrics. Find quality wooden toys such as rattles, animals, soldiers, or pull toys with wooden wheels. A cloth doll dressed in period clothing or cloth stuffed animals (no fake fur!) are wonderful. Please have a care that toys contain no small parts that can cause choking.
Special note: If your youngster has a special stuffed animal, toy or blanket that they are attached to, be sure to pack it! Even if it is not a period correct item. It will be very comforting for your child to have their beloved "binky" with them when they are tucked into their sleeping bag at night. A day in the heat doing unfamiliar things can be stressful for your child, and the presence of a familiar bed time buddy can help them feel more secure.
Despite rumors to the contrary, Civil War era folks were not as
body conscious as we have been led to believe (that was later during the Victorian
era). Nursing of children was a natural function that had to be preformed
regularly anywhere mother and child went. Remember that in these days
before birth control, women were frequently pregnant and/or frequently nursing
a babe. 
Although a woman would retire if possible to have privacy for nursing, it was not always practical. Fashion dresses opened in the front, but their tighter fit and the corset required underneath make nursing difficult. A wrapper, with it's looser fit and front opening is ideal for nursing mothers. A mother could drape a blanket over her front, or throw her apron up over her shoulder and nurse her babe with relative modesty. Of course, a nursing woman would prefer to be in the company of other ladies rather than surrounded by men or strangers...
A wealthy woman would simply hire a wet nurse and save herself from all the bother of baby feeding. In wealthy families, Nurses, Mammies or Nannies took care of all the raising of the children. The mother and father were part time parents and would resume their usual activities and occupations as soon after the birth as possible. If nursing is going to fit in with your reenacting character, you should aim for a middle class or working class role.
