Backing up the device's data to one zip file, and putting it back.
What is in a backup is an allowlist, not "whatever is in the directory": the database (the chat, what Whiskers remembers, the parents' log and the copy of every device's log, the grown-ups' choices and the day's time, and every picture, all of them rows and blobs in the one file) as a consistent copy taken with SQLite's backup interface while the app runs, and the device's service address. The app's own diagnostics and the picture cache are not data and are never read here. The device id file is left out on purpose: it names ONE device in the day's time count and in nothing else, so a backup restored onto a different device (or onto the same one after a reinstall) must not make two devices claim the same name. A restored directory has no id, so the engine makes a fresh one at its next start, and the old id's minutes stay in the household document as another device's: the day's total keeps them, nothing is counted twice. The service address is device-local and is carried as one optional entry that the shell may offer to apply; it never goes into the data directory.
The zip begins with manifest.json (format version, when, which app, every entry with its size).
Unpacking trusts nothing: the version must be one this build knows, every name must be on the
allowlist (so .., absolute paths, backslashes, drive letters and links cannot be expressed), every
entry must be in the manifest at exactly its size and no entry may be missing. It is all unpacked into
a sibling directory first; only a complete, checked tree is swapped in, and the old data is set aside
and removed only after the swap worked. A corrupt or truncated file therefore leaves the current data
exactly as it was.
Errors say what happened to the grown-up in plain words and never quote anything from a file.
The only format this build writes and the only one it reads. Format 1 held the state as JSON files and a directory of pictures; this build has no reader for it, and says so ("a form this Whiskers does not know").
The device-local service address, as UTF-8 text. Offered to the shell, never put in the data directory.
41pub const SERVICE_ADDRESS_ENTRY: &str = "device/service-address.txt";
The database, by its name in the data directory and in a backup.
43const DATABASE_ENTRY: &str = crate::DATABASE;
More than a household of a young child's pictures can come to; a manifest claiming more is refused before anything is written.
Why a backup or a restore did not happen. Every message is a sentence for a grown-up.
There is no data to back up yet.
54 NothingToBackUp,
The data on this device could not be read.
56 CannotRead,
The backup file could not be written (full storage, missing folder).
58 CannotWrite,
The file is not a Whiskers backup, or is damaged or cut short.
60 NotABackup,
Made by a newer Whiskers.
62 TooNew,
Made by a Whiskers this one does not know how to read.
64 UnknownVersion,
It holds something a backup never holds.
66 Unsafe,
It does not match its own list of contents.
68 Mismatch,
Larger than a backup can be.
70 TooLarge,
Checked fine but could not be put in place; what was on the device is untouched.
75impl std::fmt::Display for BackupError { 76 fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { 77 f.write_str(match self { 78 Self::NothingToBackUp => "There is nothing to back up yet.", 79 Self::CannotRead => "Whiskers could not read its own data, so nothing was backed up.", 80 Self::CannotWrite => "The backup could not be saved. Check that the device has room, then try again.", 81 Self::NotABackup => "That file is not a Whiskers backup, or it is damaged. Nothing was changed.", 82 Self::TooNew => "That backup was made by a newer Whiskers. Update the app first. Nothing was changed.", 83 Self::UnknownVersion => "That backup is in a form this Whiskers does not know. Nothing was changed.", 84 Self::Unsafe => "That file holds things a Whiskers backup never holds, so it was not used. Nothing was changed.", 85 Self::Mismatch => "That backup does not match its own list of contents, so it was not used. Nothing was changed.", 86 Self::TooLarge => "That backup is far larger than a Whiskers backup can be, so it was not used. Nothing was changed.", 87 Self::CannotInstall => "The backup is fine but could not be put in place. What was on this device is untouched.", 88 }) 89 } 90} 91 92impl std::error::Error for BackupError {} 93 94#[derive(Serialize, Deserialize, Debug)] 95struct Manifest { 96 format: u32, 97 created_at_ms: u64, 98 app_version: String, 99 entries: Vec<Entry>, 100} 101 102#[derive(Serialize, Deserialize, Debug, Clone)] 103struct Entry { 104 name: String, 105 size: u64, 106}
What a backup came to.
State files (the database) and the pictures in it, not counting the service address.
What a restore brought back.
The service address the backup carried, in the core's own form, for the shell to offer; None if it carried
none or one the core does not accept.
Whether an entry name may exist in a backup at all. The one gate: everything else follows from it.
Writes a backup of data_dir to the file dest (replaced whole or not at all). service_address
is carried as the optional device-local entry.
146pub fn backup_to(data_dir: &Path, dest: &Path, service_address: Option<&str>) -> Result<BackedUp, BackupError> { 147 info!("backup: starting"); 148 let db = data_dir.join(DATABASE_ENTRY); 149 if !db.is_file() { 150 info!("backup: nothing to back up"); 151 return Err(BackupError::NothingToBackUp); 152 } 153 // A consistent copy is made first, beside the destination, so the manifest's size is exactly what is 154 // written even while the app keeps writing to the database in use. 155 let copy = dest.with_file_name(format!("{}.db-copy", dest.file_name().map(|n| n.to_string_lossy().into_owned()).unwrap_or_default())); 156 // Made first, so that a destination that cannot be written to says so, and is not taken for unreadable data. 157 File::create(©).map_err(|e| { 158 error!("backup: cannot create the working copy ({:?})", e.kind()); 159 BackupError::CannotWrite 160 })?; 161 let held = whiskers_store::copy_database(&db, ©).map_err(|e| { 162 error!("backup: cannot copy the database ({e})"); 163 let _ = fs::remove_file(©); 164 BackupError::CannotRead 165 })?; 166 let result = zip_up(©, dest, service_address, held.pictures); 167 let _ = fs::remove_file(©); 168 result 169}
171fn zip_up(copy: &Path, dest: &Path, service_address: Option<&str>, pictures: u32) -> Result<BackedUp, BackupError> { 172 let size = fs::metadata(copy).map_err(|_| BackupError::CannotRead)?.len(); 173 let address = service_address.map(str::trim).filter(|a| !a.is_empty() && a.len() as u64 <= MAX_ADDRESS_BYTES); 174 let mut entries = vec![Entry { name: DATABASE_ENTRY.into(), size }]; 175 if let Some(a) = address { 176 entries.push(Entry { name: SERVICE_ADDRESS_ENTRY.into(), size: a.len() as u64 }); 177 } 178 let created_at_ms = now_ms(); 179 let manifest = Manifest { format: FORMAT_VERSION, created_at_ms, app_version: env!("CARGO_PKG_VERSION").into(), entries }; 180 let manifest_bytes = serde_json::to_vec_pretty(&manifest).map_err(|_| BackupError::CannotWrite)?; 181 182 let part = part_path(dest); 183 let result = (|| -> Result<u64, BackupError> { 184 let fail = |what: &'static str| move |e: &dyn std::fmt::Debug| { 185 error!("backup: {what} failed ({e:?})"); 186 BackupError::CannotWrite 187 }; 188 let file = File::create(&part).map_err(|e| fail("creating the file")(&e.kind()))?; 189 let mut zip = ZipWriter::new(file); 190 let w = |e: zip::result::ZipError| fail("writing the zip")(&e.to_string().len()); 191 zip.start_file(MANIFEST, options()).map_err(w)?; 192 zip.write_all(&manifest_bytes).map_err(|e| fail("writing the manifest")(&e.kind()))?; 193 zip.start_file(DATABASE_ENTRY, options()).map_err(w)?; 194 let mut src = File::open(copy).map_err(|e| { 195 error!("backup: cannot open the database copy ({:?})", e.kind()); 196 BackupError::CannotRead 197 })?; 198 let copied = io::copy(&mut Read::take(&mut src, size), &mut zip).map_err(|e| fail("copying the database")(&e.kind()))?; 199 if copied != size { 200 error!("backup: the database copy changed while it was zipped"); 201 return Err(BackupError::CannotRead); 202 } 203 if let Some(a) = address { 204 zip.start_file(SERVICE_ADDRESS_ENTRY, options()).map_err(w)?; 205 zip.write_all(a.as_bytes()).map_err(|e| fail("writing the address")(&e.kind()))?; 206 } 207 let file = zip.finish().map_err(w)?; 208 file.sync_all().map_err(|e| fail("syncing")(&e.kind()))?; 209 let bytes = file.metadata().map_err(|e| fail("measuring")(&e.kind()))?.len(); 210 fs::rename(&part, dest).map_err(|e| fail("moving into place")(&e.kind()))?; 211 Ok(bytes) 212 })(); 213 match result { 214 Ok(bytes) => { 215 info!("backup: wrote the database ({size} bytes, {pictures} pictures), {bytes} bytes in all"); 216 Ok(BackedUp { created_at_ms, files: 1, pictures, bytes }) 217 } 218 Err(e) => { 219 let _ = fs::remove_file(&part); 220 Err(e) 221 } 222 } 223} 224 225fn part_path(dest: &Path) -> PathBuf { 226 let mut name = dest.file_name().map(|n| n.to_os_string()).unwrap_or_default(); 227 name.push(".part"); 228 dest.with_file_name(name) 229}
The two filesystem steps of the swap, so a test can make the second one fail.
Replaces the contents of data_dir with the backup at src. The engine must not be running on
data_dir: the shell stops it first and restarts the process after.
249pub(crate) fn restore_with(data_dir: &Path, src: &Path, fs_ops: &dyn Rename) -> Result<Restored, BackupError> { 250 info!("restore: starting"); 251 let parent = data_dir.parent().ok_or(BackupError::CannotInstall)?; 252 let stem = data_dir.file_name().map(|n| n.to_string_lossy().into_owned()).ok_or(BackupError::CannotInstall)?; 253 let stamp = now_ms(); 254 let staging = parent.join(format!("{stem}.restoring-{stamp}")); 255 let aside = parent.join(format!("{stem}.replaced-{stamp}")); 256 257 let file = File::open(src).map_err(|e| { 258 error!("restore: cannot open the backup ({:?})", e.kind()); 259 BackupError::NotABackup 260 })?; 261 let mut zip = ZipArchive::new(file).map_err(|_| { 262 warn!("restore: not a readable zip"); 263 BackupError::NotABackup 264 })?; 265 266 let manifest = read_manifest(&mut zip)?; 267 let expected = check_manifest(&manifest)?; 268 269 // Everything is unpacked and checked before the data on the device is touched. 270 let unpacked = unpack(&mut zip, &expected, &staging); 271 let (files, service_address) = match unpacked { 272 Ok(v) => v, 273 Err(e) => { 274 let _ = fs::remove_dir_all(&staging); 275 return Err(e); 276 } 277 }; 278 // What was unpacked must be a database this build can read, or nothing is touched. 279 let pictures = match check_database(&staging) { 280 Ok(n) => n, 281 Err(e) => { 282 let _ = fs::remove_dir_all(&staging); 283 return Err(e); 284 } 285 }; 286 287 let had_old = data_dir.exists(); 288 if had_old { 289 if let Err(e) = fs_ops.rename(data_dir, &aside) { 290 error!("restore: cannot set the current data aside ({:?})", e.kind()); 291 let _ = fs::remove_dir_all(&staging); 292 return Err(BackupError::CannotInstall); 293 } 294 } 295 if let Err(e) = fs_ops.rename(&staging, data_dir) { 296 error!("restore: cannot move the restored data into place ({:?})", e.kind()); 297 if had_old { 298 if let Err(back) = fs_ops.rename(&aside, data_dir) { 299 // The one state that must be loud: the old data is at `aside`, not at `data_dir`. 300 error!("restore: COULD NOT PUT THE OLD DATA BACK ({:?}); it is kept beside the data directory", back.kind()); 301 } 302 } 303 let _ = fs::remove_dir_all(&staging); 304 return Err(BackupError::CannotInstall); 305 } 306 if had_old { 307 if let Err(e) = fs::remove_dir_all(&aside) { 308 warn!("restore: the replaced data could not be removed ({:?}); it is kept beside the data directory", e.kind()); 309 } 310 } 311 // Only an address the core accepts is offered; anything else in that entry is dropped, not shown. 312 let service_address = service_address.and_then(|t| whiskers_core::ServiceAddress::parse(t.trim()).ok()).map(|a| a.text()); 313 info!("restore: done, {files} state file(s), {pictures} pictures, address carried = {}", service_address.is_some()); 314 Ok(Restored { created_at_ms: manifest.created_at_ms, app_version: manifest.app_version, files, pictures, service_address }) 315}
Opens the unpacked database to see that it is one: a file that is not, or one from a newer Whiskers, is refused. Says how many pictures it holds.
319fn check_database(staging: &Path) -> Result<u32, BackupError> { 320 let db = staging.join(DATABASE_ENTRY); 321 let held = whiskers_store::inspect(&db).map_err(|e| { 322 warn!("restore: the database in the backup cannot be used ({e})"); 323 if e.is_newer() { BackupError::TooNew } else { BackupError::NotABackup } 324 }); 325 // Opening leaves no write-ahead log when it closes cleanly; anything left would not belong to the restored data. 326 for suffix in ["-wal", "-shm"] { 327 let _ = fs::remove_file(staging.join(format!("{DATABASE_ENTRY}{suffix}"))); 328 } 329 held.map(|h| h.pictures) 330}
332fn read_manifest(zip: &mut ZipArchive<File>) -> Result<Manifest, BackupError> { 333 if zip.is_empty() { 334 return Err(BackupError::NotABackup); 335 } 336 let mut first = zip.by_index(0).map_err(|_| BackupError::NotABackup)?; 337 if first.name() != MANIFEST || first.is_dir() || first.size() > (4 << 20) { 338 return Err(BackupError::NotABackup); 339 } 340 let mut bytes = Vec::new(); 341 first.by_ref().take(4 << 20).read_to_end(&mut bytes).map_err(|_| BackupError::NotABackup)?; 342 // The version is read before the rest, so a newer format's other changes cannot read as "damaged". 343 #[derive(Deserialize)] 344 struct Version { 345 format: u32, 346 } 347 let version: Version = serde_json::from_slice(&bytes).map_err(|_| BackupError::NotABackup)?; 348 match version.format { 349 FORMAT_VERSION => {} 350 0 => return Err(BackupError::NotABackup), 351 v if v > FORMAT_VERSION => return Err(BackupError::TooNew), 352 _ => return Err(BackupError::UnknownVersion), 353 } 354 serde_json::from_slice(&bytes).map_err(|_| BackupError::NotABackup) 355}
The manifest's entries by name, after every rule that needs only the manifest.
358fn check_manifest(m: &Manifest) -> Result<BTreeMap<String, u64>, BackupError> { 359 if m.entries.len() > MAX_ENTRIES { 360 return Err(BackupError::TooLarge); 361 } 362 let mut by_name = BTreeMap::new(); 363 let mut total: u64 = 0; 364 for e in &m.entries { 365 if !allowed(&e.name) { 366 warn!("restore: the manifest lists a name a backup never holds"); 367 return Err(BackupError::Unsafe); 368 } 369 if by_name.insert(e.name.clone(), e.size).is_some() { 370 return Err(BackupError::Mismatch); 371 } 372 total = total.saturating_add(e.size); 373 } 374 if total > MAX_TOTAL_BYTES { 375 return Err(BackupError::TooLarge); 376 } 377 if by_name.get(SERVICE_ADDRESS_ENTRY).is_some_and(|s| *s > MAX_ADDRESS_BYTES) { 378 return Err(BackupError::Unsafe); 379 } 380 Ok(by_name) 381}
383fn unpack(zip: &mut ZipArchive<File>, expected: &BTreeMap<String, u64>, staging: &Path) -> Result<(u32, Option<String>), BackupError> { 384 let cannot_stage = |e: io::Error| { 385 error!("restore: cannot write the staging directory ({:?})", e.kind()); 386 BackupError::CannotInstall 387 }; 388 fs::create_dir_all(staging).map_err(cannot_stage)?; 389 let mut seen: BTreeSet<String> = BTreeSet::new(); 390 let mut files = 0u32; 391 let mut address = None; 392 if zip.len() != expected.len() + 1 { 393 return Err(BackupError::Mismatch); 394 } 395 for i in 1..zip.len() { 396 let mut entry = zip.by_index(i).map_err(|_| BackupError::NotABackup)?; 397 let name = entry.name().to_owned(); 398 if entry.is_dir() || entry.is_symlink() || !allowed(&name) { 399 warn!("restore: an entry is a directory, a link or has a name a backup never holds"); 400 return Err(BackupError::Unsafe); 401 } 402 let Some(&size) = expected.get(&name) else { return Err(BackupError::Mismatch) }; 403 if !seen.insert(name.clone()) || entry.size() != size { 404 return Err(BackupError::Mismatch); 405 } 406 if name == SERVICE_ADDRESS_ENTRY { 407 let mut text = String::new(); 408 entry.by_ref().take(size + 1).read_to_string(&mut text).map_err(|_| BackupError::NotABackup)?; 409 if text.len() as u64 != size { 410 return Err(BackupError::Mismatch); 411 } 412 address = Some(text); 413 continue; 414 } 415 let out_path = staging.join(&name); 416 let mut out = File::create(&out_path).map_err(cannot_stage)?; 417 // One byte past the declared size, so an entry that lies about its size is seen, not truncated. 418 let copied = io::copy(&mut entry.by_ref().take(size + 1), &mut out).map_err(|e| { 419 warn!("restore: an entry could not be read ({:?})", e.kind()); 420 BackupError::NotABackup 421 })?; 422 if copied != size { 423 return Err(BackupError::Mismatch); 424 } 425 out.sync_all().map_err(cannot_stage)?; 426 files += 1; 427 } 428 if seen.len() != expected.len() { 429 return Err(BackupError::Mismatch); 430 } 431 if files == 0 { 432 // An address and nothing else is not a backup of anything. 433 return Err(BackupError::Mismatch); 434 } 435 Ok((files, address)) 436} 437 438#[cfg(test)] 439mod tests;