좋은 글 감사합니다.
위와 같은 방법으로 R환경을 띄우면 3.X 버전이 설치되는 경우가 있는데 그럴 경우 수동으로 환경을 생성한 후 R을 설치하면 4.X버전을 사용할 수 있습니다.

--  Anaconda Prompt에서
> conda create --name your-env-name
> conda activate your-env-name
(your-env-name)> conda install -c conda-forge r-base
(your-env-name)> conda install -c conda-forge/label/gcc7 r-base
(your-env-name)> conda install jupyter
(your-env-name)> R
(R 실행 후, R에서)
R>  install.packages('IRkernel')
R>  IRkernel::installspec()
R> q()
(R 종료, 다시 Anaconda Prompt)
(your-env-name)> jupyter notebook

your-env-name에 생성하고자 하시는 환경 이름을 넣어주시면 됩니다.

원문) https://datascience.stackexchange.com/questions/77335/anconda-r-version-how-to-upgrade-to-4-0-and-later