![]() However, we can directly verify that ReLU is globally convex as well by substituting the definition into the above inequality. Bring it all together, and you have your graph This tutorial shows you the entire process for graphing a piecewise linear function. Thus, ReLU is (at least) piecewise convex. Linear functions (in both senses) are convex, and so piecewise linear implies piecewise convex. So, to directly answer your question, "piecewise linear" should not be interpreted as "nonlinear" in general, but there are piecewise linear functions which are nonlinear (and those which are linear).Īs with linearity, we can discuss convexity in both global and piecewise contexts.Ī function is called convex on some domain when every pair of points $x,y$ in that domain has Obviously, any linear function is piecewise linear, but the same is not true in reverse, as we see here with ReLU. To agree with the left half-line, we need $f(x) \equiv 0$ for the right half-line, we need $f(x) = x$.īecause there is a partition of the line into intervals such that ReLU is linear (in both senses) on the interior of each interval, we say that ReLU is piecewise linear. In this case, the vector space and the underlying field are both $\Bbb R$, and this implies the formĪny attempt to put ReLU into this form will result in a function which is only valid on a half-line. ![]() It is a piecewise-defined function whose pieces are linear. ![]() We say a function is linear iffįor $x,y$ from some vector space and $\alpha$ from the underlying field. In mathematics, a piecewise linear function is a function composed of straight-line sections. The second definition comes from linear algebra, and is a restriction on this class of functions. (as Brian Borchers points out in the comments, this type of function is usually called affine beyond elementary algebra to avoid conflict with the second definition below) The first definition most are exposed to is that a linear function is polynomial of degree 1, i.e. There are two competing definitions for a linear function $f:\Bbb R\mapsto\Bbb R$, but neither of them would allow ReLU.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |