【Linux 使用】之–Arch Linux 下使用 rclone 挂载 onedrive 备份数据

#Linux #教程 #工具 #系统 #Ubuntu #arch #archlinux

最近需要整理大量的零星资料，并且要求按天打包备份，本文使用 rclone 实现 OneDrive 挂在到本地，并通过 crontab 定时任务备份数据到挂载盘，可以有效解决资料频繁备份和小硬盘的问题。

1. 挂载 OneDrive

1.1. 获取 OneDrive Token

安装rclone

sudo pacman -S rclone

登陆账号，获取 OneDrive Token 字符串

rclone authorize "onedrive"

字符串格式，如下

{"access_token":"eXAiOiJKV1xxxxxxxxxx（此处省略N个字符）Rpw","expiry":"2023-03-17T17:33:43.569356467+08:00"}

1.2. Rclone 配置

新建 rclone 配置文件

rclone config

按照如下提示一步一步配置

arch :~$ rclone config
No remotes found, make a new one?
n) New remote
s) Set configuration password
q) Quit config
n/s/q> n      # 输入n新建
name> od      # 名字od，也可以自己取名字，后面作相应修改
Option Storage.
Type of storage to configure.
Choose a number from below, or type in your own value.
 1 / 1Fichier
   \ (fichier)
 2 / Akamai NetStorage
   \ (netstorage)
 3 / Alias for an existing remote
   \ (alias)
 4 / Amazon Drive
   \ (amazon cloud drive)
 5 / Amazon S3 Compliant Storage Providers including AWS, Alibaba, Ceph, Digital Ocean, Dreamhost, IBM COS, Lyve Cloud, Minio, RackCorp, SeaweedFS, and Tencent COS
   \ (s3)
 6 / Backblaze B2
   \ (b2)
 7 / Better checksums for other remotes
   \ (hasher)
 8 / Box
   \ (box)
 9 / Cache a remote
   \ (cache)
10 / Citrix Sharefile
   \ (sharefile)
11 / Compress a remote
   \ (compress)
12 / Dropbox
   \ (dropbox)
13 / Encrypt/Decrypt a remote
   \ (crypt)
14 / Enterprise File Fabric
   \ (filefabric)
15 / FTP Connection
   \ (ftp)
16 / Google Cloud Storage (this is not Google Drive)
   \ (google cloud storage)
17 / Google Drive
   \ (drive)
18 / Google Photos
   \ (google photos)
19 / Hadoop distributed file system
   \ (hdfs)
20 / Hubic
   \ (hubic)
21 / In memory object storage system.
   \ (memory)
22 / Jottacloud
   \ (jottacloud)
23 / Koofr, Digi Storage and other Koofr-compatible storage providers
   \ (koofr)
24 / Local Disk
   \ (local)
25 / Mail.ru Cloud
   \ (mailru)
26 / Mega
   \ (mega)
27 / Microsoft Azure Blob Storage
   \ (azureblob)
28 / Microsoft OneDrive
   \ (onedrive)
29 / OpenDrive
   \ (opendrive)
30 / OpenStack Swift (Rackspace Cloud Files, Memset Memstore, OVH)
   \ (swift)
31 / Pcloud
   \ (pcloud)
32 / Put.io
   \ (putio)
33 / QingCloud Object Storage
   \ (qingstor)
34 / SSH/SFTP Connection
   \ (sftp)
35 / Sia Decentralized Cloud
   \ (sia)
36 / Storj Decentralized Cloud Storage
   \ (storj)
37 / Sugarsync
   \ (sugarsync)
38 / Transparently chunk/split large files
   \ (chunker)
39 / Union merges the contents of several upstream fs
   \ (union)
40 / Uptobox
   \ (uptobox)
41 / Webdav
   \ (webdav)
42 / Yandex Disk
   \ (yandex)
43 / Zoho
   \ (zoho)
44 / http Connection
   \ (http)
45 / premiumize.me
   \ (premiumizeme)
46 / seafile
   \ (seafile)
Storage> 28     # 选择28 / Microsoft OneDrive
Option client_id.
OAuth Client Id.
Leave blank normally.
Enter a value. Press Enter to leave empty.
client_id>      # 不用填，直接回车
Option client_secret.
OAuth Client Secret.
Leave blank normally.
Enter a value. Press Enter to leave empty.
client_secret>  # 不用填，直接回车
Option region.
Choose national cloud region for OneDrive.
Choose a number from below, or type in your own string value.
Press Enter for the default (global).
 1 / Microsoft Cloud Global
   \ (global)
 2 / Microsoft Cloud for US Government
   \ (us)
 3 / Microsoft Cloud Germany
   \ (de)
 4 / Azure and Office 365 operated by 21Vianet in China
   \ (cn)
region> 1      # 选 1 / Microsoft Cloud Global全球，依据自己的情况来定
Edit advanced config?
y) Yes
n) No (default)
y/n> n         # 高级设置不需要
Use auto config?
 * Say Y if not sure
 * Say N if you are working on a remote or headless machine

y) Yes (default)
n) No
y/n> n         # 自己设置
Option config_token.
For this to work, you will need rclone available on a machine that has
a web browser available.
For more help and alternate methods see: https://rclone.org/remote_setup/
Execute the following on the machine with the web browser (same rclone
version recommended):
        rclone authorize "onedrive"
Then paste the result.
Enter a value.
config_token>          # 填入获取的OneDrive Token
{"access_token":"eXAiOiJKV1xxxxxxxxxx（此处省略N个字符）Rpw","expiry":"2023-03-17T17:33:43.569356467+08:00"}
Option config_type.
Type of connection
Choose a number from below, or type in an existing string value.
Press Enter for the default (onedrive).
 1 / OneDrive Personal or Business
   \ (onedrive)
 2 / Root Sharepoint site
   \ (sharepoint)
   / Sharepoint site name or URL
 3 | E.g. mysite or https://contoso.sharepoint.com/sites/mysite
   \ (url)
 4 / Search for a Sharepoint site
   \ (search)
 5 / Type in driveID (advanced)
   \ (driveid)
 6 / Type in SiteID (advanced)
   \ (siteid)
   / Sharepoint server-relative path (advanced)
 7 | E.g. /teams/hr
   \ (path)
config_type> 1     # 选 1 / OneDrive Personal or Business
Drive OK?

Found drive "root" of type "business"
URL: https://zylvt-my.sharepoint.com/personal/alongle_zylvt_onmicrosoft_com/Documents

y) Yes (default)
n) No
y/n> y             # 使用获取的默认设置
--------------------
[od]
type = onedrive
token = 
{"access_token":"eXAiOiJKV1xxxxxxxxxx（此处省略N个字符）Rpw","expiry":"2022-04-20T18:16:18.5766008+08:00"}
drive_id = b!XqBSqnPmj06ysrTEjqra6-7pdEqzcnVNlC0vu-5VZHCDi_Ove7VrTJX_h_i5jJrg
drive_type = business
--------------------
y) Yes this is OK (default)
e) Edit this remote
d) Delete this remote
y/e/d> y           # 确认
Current remotes:

Name                 Type
====                 ====
od                   onedrive

e) Edit existing remote
n) New remote
d) Delete remote
r) Rename remote
c) Copy remote
s) Set configuration password
q) Quit config
e/n/d/r/c/s/q> q   # 设置完毕退出

查看配置文件

arch :~$ more ./. Config/rclone/rclone. Conf
[od]
Type = onedrive
token = {"access_token":"eXAiOiJKV1xxxxxxxxxx（此处省略N个字符）Rpw","expiry":"2023-03-17T17:33:43.569356467+08:00"}
drive_id = b!XqBSqnPmj06ysrTEjqra6-7pdEqzcnVNlC0vu-5VZHCDi_Ove7VrTJX_h_i5jJrg
drive_type = business

在当前用户下创建 onedrive 本地挂载目录

mkdir /home/$USER/onedrive

挂载 onedrive

command="mount od:/ /home/$USER/onedrive --attr-timeout 5m --vfs-cache-mode full --vfs-cache-max-age 24h --vfs-cache-max-size 20G --vfs-read-chunk-size-limit 100M --buffer-size 100M --umask 000"

参数说明：

Od:/file：od 为配置好的 onedrive，od:/file 是 onedrive 路径
/home/onedrive：本地文件夹路径
–allow-other：允许非当前 rclone 用户外其它用户进行访问，当前用户使用不需要。
–attr-timeout 5 m：文件属性缓存，（大小，修改时间等）的时间。如果 VPS 配置比较低，建议适当提高这个值，避免过多内核交互，降低资源占用。
-vfs-cache-mode full：开启 VFS 文件缓存，可减少 rclone 与 API 交互，同时可提高文件读写效率
–vfs-cache-max-age 24 h：VFS 文件缓存时间，这里设置 24 小时，如果文件很少更改，建议设置更长时间
–vfs-cache-max-size 20 G：VFS 文件缓存上限大小，建议不超过当前空余磁盘的 50%
Vfs-read-chunk-size-limit 100 M：分块读取大小，这里设置的是 100 M，可提高文件读的效率，比如 1 G 的文件，大致分为 10 个块进行读取，但与此同时 API 请求次数也会增多
–buffer-size 100 M：内存缓存，如果您内存比较小，可降低此值，如果内存比较大，可适当提高

1.3. 设置 rclone 开机自启服务

创建 rclone. Service 启动服务

cat > /home/$USER/.config/systemd/user/rclone.service <<EOF
[Unit]
Description=Rclone
After=network-online.target

[Service]
Type=simple
ExecStart=$(command -v rclone) ${command}
Restart=on-abort

[Install]
WantedBy=default.target
EOF

配置生效

# 重新加载服务
systemctl --user daemon-reload

# rclone 服务开启开机自启
systemctl --user enable rclone

# 启动 rclone 服务
systemctl --user start rclone

显示磁盘空间信息，查看是否挂载成功

df -h

arch :~$ df -h
文件系统        大小  已用  可用 已用% 挂载点dev             3.7G     0  3.7G    0% /dev
run             3.7G  1.7M  3.7G    1% /run
/dev/sdb7        52G   40G  9.7G   81% /
tmpfs           3.7G  329M  3.4G    9% /dev/shm
/dev/sdb7        52G   40G  9.7G   81% /home
tmpfs           3.7G   66M  3.7G    2% /tmp
/dev/sdb6       511M  312K  511M    1% /boot/efi
tmpfs           757M  100K  757M    1% /run/user/1000
od:             5.0T  1.9T  3.2T   38% /home/arch/onedrive

取消挂载的相关操作

fusermount -qzu od # 取消挂载名为od的磁盘
systemctl --user stop rclone # 停止rclone服务
systemctl --user disable rclone # 关闭rclone服务开机自启
rclone mount od:/ /home/$USER/onedrive --attr-timeout 5m --vfs-cache-mode full --vfs-cache-max-age 24h --vfs-cache-max-size 10G --vfs-read-chunk-size-limit 100M --buffer-size 100M --daemon # rclone临时挂载，重启服务器后失效

2. 设置备份脚本

cd ~
vim backup.sh

脚本如下:

#!/bin/bash

# 备份文件路径
LocalBackDir=/home/$USER/syncdir

# 备份文件存放路径
RemoteBackDir=/home/$USER/onedrive/data_backups/archsyncdir

# 当前系统日期
DATE=`date +"%Y-%m-%d-%H%M%S"`

# Log存放路径
LogFile=./backup.logs

# 新建日志文件
touch $LogFile

# 文件名称
BackupFile="archsyncdir-$DATE"

# 输出日志。
echo "------------------------Start-------------------------------" >> $LogFile

tar -cpzPf $BackupFile.tgz --exclude=./$BackupFile.tgz $LocalBackDir

echo "备份文件已创建。" >> $LogFile

# 查找本地备份目录下，时间为60分钟之内，并且后缀为.tgz的备份文件
BACKUPFILE_SEND_TO_REMOTE=$(find ./ -maxdepth 1 -type f -mmin -60  -name '*.tgz')

# 追加日志。
echo "文件名称: $BACKUPFILE_SEND_TO_REMOTE At $(date +"%Y-%m-%d %H:%M:%S")" >>  $LogFile

# 备份到远程服务器
cp $BACKUPFILE_SEND_TO_REMOTE $RemoteBackDir

# 输出日志：备份的文件名
echo "文件已拷贝完成: $BACKUPFILE_SEND_TO_REMOTE。" >> $LogFile

rm -rf $BACKUPFILE_SEND_TO_REMOTE

# 追加日志。
echo "清理本机备份文件。" >> $LogFile

echo "------------------------End---------------------------------" >> $LogFile

测试一下

sh backup.sh

# 查看一下日志
cat backup.logs

# 输出内容如下，表示成功
------------------------Start-------------------------------  
备份文件已创建。  
文件名称: ./archsyncdir-2023-03-18-083159.tgz At 2023-03-18 08:32:07  
文件已拷贝完成: ./archsyncdir-2023-03-18-083159.tgz。  
清理本机备份文件。  
------------------------End---------------------------------

2.1. 通过 crontab 定时执行

给予脚本执行权限

chmod +x backup.sh

安装cronie

sudo pacman -S cronie

启动服务

sudo systemctl enable --now cronie.service

# 编辑任务
crontab -e

# 增加一条任务，3:00 执行/home/$USER/backup.sh脚本
0 3 * * * /home/$USER/backup.sh

2.2. crontab 用法

具体含义如下：

# Example of job definition:
# .---------------- 分 (0 - 59)
# |  .------------- 时 (0 - 23)
# |  |  .---------- 日 (1 - 31)
# |  |  |  .------- 月 (1 - 12) OR jan,feb,mar,apr ...
# |  |  |  |  .---- 星期 (0 - 6) (Sunday=0 or 7) OR sun,mon,tue,wed,thu,fri,sat
# |  |  |  |  |
# *  *  *  *  * command to be executed

空格用来分开字段，也可以用下面特殊字符来设定范围:

符号	描述
`*`	通配符，表示所有支持的时间值
`,`	逗号，分隔多个时间
`-`	连接两个数值，给出一个范围
`/`	指定一个周期或频率

例如：

# 将会在周内从早上 9 点到下午 4 点 55 分，每隔 5 分钟执行一次脚本 /home/$USER/backup.sh，除了 6 、7、8 月
*/5 9-16 * 1-5,9-12 1-5 /home/$USER/backup.sh

此外，crontab 还有一些特殊的关键字。

关键字	描述
@reboot	启动时
@yearly	每年一次
@annually	同 @yearly
@monthly	每月一次
@weekly	每周一次
@daily	每天一次
@midnight	午夜，同 @daily
@hourly	每小时一次

例如：

# 将在启动时执行脚本 /home/$USER/backup.sh
@reboot /home/$USER/backup.sh

crontab 基本命令

# crontab用法
Usage:  
crontab [options] file  
crontab [options]  
crontab -n [hostname]  
  
Options:  
-u <user>  define user 指定编辑用户，未指定默认当前用户
-e         edit user's crontab 编辑用户 crontabs
-l         list user's crontab 查看用户 crontabs
-r         delete user's crontab 移除用户 crontabs
-i         prompt before deleting  
-n <host>  set host in cluster to run users' crontabs  
-c         get host in cluster to run users' crontabs  
-T <file>  test a crontab file syntax  
-V         print version and exit  
-x <mask>  enable debugging  
  
Default operation is replace, per 1003.2

参考： https://wiki.archlinuxcn.org/wiki/Cron

目录CONTENT

【Linux使用】之--Arch Linux下使用rclone挂载onedrive备份数据