A family of random variables {X(θ)} parameterized by the parameter θ satisfies stochastic convexity (SCX) if and only if for any increasing and convex function f(x), Ef[X(θ)] is convex in θ. This definition, however, has a major drawback for the lack of certain important closure properties. In this paper we establish the notion of strong stochastic convexity (SSCX), which implies SCX. We demonstrate that SSCX is a property enjoyed by a wide range of random variables. We also show that SSCX is preserved under random mixture, random summation, and any increasing and convex operations that are applied to a set of independent random variables. These closure properties greatly facilitate the study of parametric convexity of many stochastic systems. Applications to GI/G/1 queues, tandem and cyclic queues, and tree-like networks are discussed. We also demonstrate the application of SSCX in bounding the performance of certain systems.