Side channel attacks are variably understood as excluding or including fault attacks. Let's start with the excluding definition, where side channel attacks monitor a device running some security-critical process, without deliberately attempting to alter its normal operation.
The most straightforward and perhaps earliest form of side channel attack is probing the internals of the circuit using contacts on the physical traces (e.g. of a PCB); that's called micro-probing when done directly on an IC. The archetypal goal of probing is capturing the data bus of a microprocessor system, letting one examine all that it manipulates. Some devices have dedicated sensors to detect probing; see e.g. figures 2 and 3 in Ross Anderson, Mike Bond, Jolyon Clulow, Sergei Skorobogatov Cryptographic Processors - A Survey, in proceedings or IEEE, 2006. On occasion, micro-probing indirectly cause misbehavior and can be detected as some fault attacks are in software, see last 5 bullets.
Many side channel attacks (including most with the keyword Differential) require many cryptographic computations to be performed. Counting the number of computations (with a carefully implemented power-loss-resistant counter in EEPROM or Flash) and limiting that number is a from of detection, and is effective if a small enough limit is functionally acceptable. Alternatively, the key can change regularly (e.g. every 100 uses), but typically that must be integrated in the specifications, and recovery in case of protocol errors can be an issue; and that's prevention rather than detection as asked.
In the special case of Power Analysis, it conceivably could be detected that a test setup inserts a resistor on Vcc or Gnd (the simplest option for a Power Analysis setup), and thus that the power supply voltage (which the device can measure) changes with DC power consumption (that the device can change by turning on or off some circuits, or changing a frequency divider). The effective series resistance of the power supply could thus be estimated to some degree, and attack suspected above some threshold. However, doing this has a risk of false positive (e.g. if the power contacts are dirty); and it could be circumvented by a better attack setup. I do not know that this line of defense is used.
Other forms of side-channel attacks seem basically undetectable to me. One can not hope to test if something is timing execution, monitoring EMI emission..
Fault attacks are detectable, high-security ICs (like those in many Smart Cards) have sensors for that, and high-security software has countermeasures trying to block these attacks by performing redundant checks. There are things like:
- Vcc out-of-range or/and rapid change sensor.
- Light sensor, to detect optical flash or laser pulses, as well as careless use of the IC in lighted environment after epoxy removal or de-passivation.
- Clk frequency out-of-bounds detector (glitching Clk used to be an effective fault attack in early Smart Cards; but the trend is internally generated clock).
- Temperature-out-of-bound sensor.
- Mesh(es) and associated sensors: one ore more mesh(es) (a long wired forming a regular pattern) is/are added on top of the IC or circuit, and it is hopped that probing will interfere with the mesh, activating some sensor: mesh-has-been-opened, meshes-have-been-shorted, capacitance-between-meshes surveillance (which has a chance to detect de-passivation). See this archived FlyLogic blog, read after Mesh Comparison.
- Internal self-test: like stuck TRNG detector, parity check on bus or memory, or even dual CPUs performing the same operations concurrently with their result matched.
- Checking results of cryptographic computations; the archetypal one is checking that $x\equiv y^e\pmod N$ after having computed $y=x^d\bmod N$ in RSA (especially if the computation of $y$ was using the CRT).
- Redundant implementation and testing of critical states, like are the access rights granted by a proper PIN presentation.
- Checksums on critical variables or code.
- I imagine, much more..
Note: I have not much touched prevention (rather than detection) of Side Channel and Fault attack, which is a much wider subject.