This work is dealing with the use of miscanthus, recycled textile and rice husks as reinforcement for chitosan matrix to elaborate new insulating composites for building application. Insulating composites having thermal conductivity of 0.07–0.09 W.m−1.K−1 and density of 350–400 Kg.m−3 were manufactured by thermocompression. Different granulometry of miscanthus (0.2–0.5 cm and 1–2 cm) and rice husks (1–2 cm) have been used with and without textile to evaluate the effect of reinforcements particle size and nature on composites thermal and mechanical properties. Thermal conductivity and effusivity shows a linear behavior related to their increasing by raising up the reinforcement’s particle size. The highest mechanical properties in bending (modulus: 69–65 MPa; stress: 0.48–0.45 MPa) and compression (modulus: 36–26 MPa; stress: 0.65–0.56 MPa) were found for the formulations with small size miscanthus. Thus, the incorporation of small miscanthus particle size <1 mm leads to satisfying and promising results in terms of composites competing with the conventional insulating materials used nowadays.