Boards of directors, especially when a company forms, should be kept small. Remember that a board of directors has the ultimate control over the management of the company and represents the company's stockholders. Therefore, when a company is formed, it often makes sense to just have the founder(s) on the board of directors. If there are non-founders whose guidance would be valuable to the company, consider bringing them on as an advisor rather than as a director; this way you can receive their guidance without ceding control.
Most investors, especially venture investors, will insist on at least one board seat for each round of investment (e.g., a Series A company may have one investor director, while a Series C company may have three investor directors). That is one reason that it's important to only take investors with whom you are able to work well, and who have experience growing businesses similar to yours. Venture capitalists often have lots of great board experience, but they will only work well on your board if they work well with you.