Softplus

In mathematics and machine learning, the softplus function is

${\displaystyle f(x)=\log(1+e^{x}).}$

It is a smooth approximation (in fact, an analytic function) to the ramp function, which is known as the rectifier or ReLU (rectified linear unit) in machine learning. For large negative ${\displaystyle x}$ it is ${\displaystyle \log(1+e^{x})=\log(1+\epsilon )\gtrapprox \log 1=0}$, so just above 0, while for large positive ${\displaystyle x}$ it is ${\displaystyle \log(1+e^{x})\gtrapprox \log(e^{x})=x}$, so just above ${\displaystyle x}$.

The names softplus^[1][2] and SmoothReLU^[3] are used in machine learning. The name "softplus" (2000), by analogy with the earlier softmax (1989) is presumably because it is a smooth (soft) approximation of the positive part of x, which is sometimes denoted with a superscript plus, ${\displaystyle x^{+}:=\max(0,x)}$.