The function $G'$, as you intuited, is indeed a PRG. The standard way to establish such results is through the hybrid argument. Details follow.
Let $G_n:\{0,1\}^n\rightarrow\{0,1\}^{n+1}$ be an $\epsilon(n)$-secure PRG; that is:
$$G_n(U_n)\approx_{\epsilon(n)}U_{n+1},~~~~~(1)$$
where $\approx_{\epsilon(n)}$ denotes computational distance$^*$ of $\epsilon(n)$, and $U_n$ is the uniform distribution on $\{0,1\}^n$.
Given $(1)$, we show that $G'_n$ is an $2\epsilon(n/2)$-secure PRG (for even $n$ for the moment); that is:
$$G_{n/2}(U_{n/2})\|G_{n/2}(U_{n/2})\approx_{2\epsilon(n/2)}U_{n+2}.~~~~~(2)$$
This is accomplished in two steps using a hybrid (intermediate) distribution $U_{n/2+1}\|G_{n/2}(U_{n/2})$. We first show that
$$G_{n/2}(U_{n/2})\|G_{n/2}(U_{n/2})\approx_{\epsilon(n/2)}U_{n/2+1}\|G_{n/2}(U_{n/2}),~~~~~(3)$$
and then that
$$U_{n/2+1}\|G_{n/2}(U_{n/2})\approx_{\epsilon(n/2)}U_{n+2}.~~~~~(4)$$
$(2)$ follows by an application of the triangle inequality to $(3)$ and $(4)$. Since the arguments for $(3)$ and $(4)$ are similar, we prove just $(3)$.
Suppose for contradiction that $G_{n/2}(U_{n/2})\|G_{n/2}(U_{n/2})$ is not $\epsilon(n/2)$-close to $U_{n/2+1}\|G_{n/2}(U_{n/2})$; then we show that $G_{n/2}(U_{n/2})$ is not $\epsilon(n/2)$-close to $U_{n/2}$ contradicting $(1)$. To be precise, given a distinguisher $\mathsf{D}$ for the former, we construct a distinguisher $\mathsf{D}'$ for the latter. The construction of $\mathsf{D}'$ is straightforward: given a challenge $y\in\{0,1\}^{n/2+1}$ (which is either $G_{n/2}(U_{n/2})$ or $U_{n/2+1}$), run $\mathsf{D}$ on $y\|G_{n/2}(U_{n/2})$ and return whatever $\mathsf{D}$ returns. If $y=G_{n/2}(U_{n/2})$ then $\mathsf{D}'$ presents $G_{n/2}(U_{n/2})\|G_{n/2}(U_{n/2})$ to $\mathsf{D}$; otherwise, it presents $U_{n/2+1}\|G_{n/2}(U_{n/2})$. Thus, $\mathsf{D}$'s ability to distinguish the distributions in $(3)$ helps $\mathsf{D}'$ distinguish the distributions in $(1)$.
A similar argument can be made for odd values of $n$, and together you get that $G'$ is a PRG. The argument can be extended for $G'(s)=G(s_1)||G(s_2)||\ldots G(s_m)$ for $s=s_1||s_2||\ldots s_m$: one has to define a $m-1$ hybrid distributions, where the $i^{th}$ hybrid replaces the first $i$ calls of $G$ with a uniformly random string; i.e., assuming $m$ divides $n$,
$$\underbrace{U_{n/m+1}\|\cdots\|U_{n/m+1}}_{i \text{ terms}}\|G_{n/m}(U_{n/m})\|\cdots\|G_{n/m}(U_{n/m}).$$
Also, this PRG is only $m\cdot\epsilon(n/m)$-secure.
$^*$A distribution $X=\{X_n\}$ is $\epsilon(n)$-computationally-close from another distribution $Y=\{Y_n\}$ if for every computationally-bounded distinguisher $\mathsf{D}$,
$$\big|\Pr[\mathsf{D}(X_n)=1]-\Pr[\mathsf{D}(Y_n)=1]\big|\leq\epsilon(n).$$
In particular $X$ is computationally indistinguishable from $Y$ if $\epsilon(n)$ is negligible in $n$.
