linux-api.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Gabriel Krisman Bertazi <krisman@collabora.com>
To: jack@suse.com, amir73il@gmail.com
Cc: djwong@kernel.org, tytso@mit.edu, dhowells@redhat.com,
	khazhy@google.com, linux-fsdevel@vger.kernel.org,
	linux-ext4@vger.kernel.org, linux-api@vger.kernel.org,
	repnop@google.com,
	Gabriel Krisman Bertazi <krisman@collabora.com>,
	kernel@collabora.com, Dave Chinner <david@fromorbit.com>
Subject: [PATCH v7 00/28] file system-wide error monitoring
Date: Thu, 14 Oct 2021 18:36:18 -0300	[thread overview]
Message-ID: <20211014213646.1139469-1-krisman@collabora.com> (raw)

Hi,

This attempts to get the ball rolling again for the FAN_FS_ERROR.  This
version is slightly different from the previous approaches, since it uses
mempool for memory allocation, as suggested by Jan.  It has the
advantage of simplifying a lot the enqueue/dequeue, which is now much
more similar to other event types, but it also means the guarantee that
an error event will be available is diminished.

The way we propagate superblock errors also changed. Now we use
FILEID_ROOT internally, and mangle it prior to copy_to_user.

I am no longer sure how to guarantee that at least one mempoll slot will
be available for each filesystem.  Since we are now tying the poll to
the entire group, a stream of errors in a single file system might
prevent others from emitting an error.  The possibility of this is
reduced since we merge errors to the same filesystem, but it is still
possible that they occur during the small window where the event is
dequeued and before it is freed, in which case another filesystem might
not be able to obtain a slot.

I'm also creating a poll of 32 entries initially to avoid spending too
much memory.  This means that only 32 filesystems can be watched per
group with the FAN_FS_ERROR mark, before fanotify_mark starts returning
ENOMEM.

This was tested with LTP for regressions and also using the sample code
on the last patch, with a corrupted image.  I wrote a new ltp test for
this feature which is being reviewed and is available at:

  https://gitlab.collabora.com/krisman/ltp  -b fan-fs-error

In addition, I wrote a man-page that can be pulled from:

  https://gitlab.collabora.com/krisman/man-pages.git -b fan-fs-error

And is being reviewed at the list.

I also pushed this full series to:

  https://gitlab.collabora.com/krisman/linux -b fanotify-notifications-single-slot

Thank you

Original cover letter
---------------------
Hi,

This series follow up on my previous proposal [1] to support file system
wide monitoring.  As suggested by Amir, this proposal drops the ring
buffer in favor of a single slot associated with each mark.  This
simplifies a bit the implementation, as you can see in the code.

As a reminder, This proposal is limited to an interface for
administrators to monitor the health of a file system, instead of a
generic inteface for file errors.  Therefore, this doesn't solve the
problem of writeback errors or the need to watch a specific subtree.

In comparison to the previous RFC, this implementation also drops the
per-fs data and location, and leave those as future extensions.

* Implementation

The feature is implemented on top of fanotify, as a new type of fanotify
mark, FAN_ERROR, which a file system monitoring tool can register to
receive error notifications.  When an error occurs a new notification is
generated, in addition followed by this info field:

 - FS generic data: A file system agnostic structure that has a generic
 error code and identifies the filesystem.  Basically, it let's
 userspace know something happened on a monitored filesystem.  Since
 only the first error is recorded since the last read, this also
 includes a counter of errors that happened since the last read.

* Testing

This was tested by watching notifications flowing from an intentionally
corrupted filesystem in different places.  In addition, other events
were watched in an attempt to detect regressions.

Is there a specific testsuite for fanotify I should be running?

* Patches

This patchset is divided as follows: Patch 1 through 5 are refactoring
to fsnotify/fanotify in preparation for FS_ERROR/FAN_ERROR; patch 6 and
7 implement the FS_ERROR API for filesystems to report error; patch 8
add support for FAN_ERROR in fanotify; Patch 9 is an example
implementation for ext4; patch 10 and 11 provide a sample userspace code
and documentation.

I also pushed the full series to:

  https://gitlab.collabora.com/krisman/linux -b fanotify-notifications-single-slot

[1] https://lwn.net/Articles/854545/
[2] https://lwn.net/Articles/856916/

Cc: Darrick J. Wong <djwong@kernel.org>
Cc: Theodore Ts'o <tytso@mit.edu>
Cc: Dave Chinner <david@fromorbit.com>
Cc: jack@suse.com
To: amir73il@gmail.com
Cc: dhowells@redhat.com
Cc: khazhy@google.com
Cc: linux-fsdevel@vger.kernel.org
Cc: linux-ext4@vger.kernel.org
Cc: linux-api@vger.kernel.org
Cc: linux-api@vger.kernel.org

Amir Goldstein (3):
  fsnotify: pass data_type to fsnotify_name()
  fsnotify: pass dentry instead of inode data
  fsnotify: clarify contract for create event hooks

Gabriel Krisman Bertazi (25):
  fsnotify: Don't insert unmergeable events in hashtable
  fanotify: Fold event size calculation to its own function
  fanotify: Split fsid check from other fid mode checks
  inotify: Don't force FS_IN_IGNORED
  fsnotify: Add helper to detect overflow_event
  fsnotify: Add wrapper around fsnotify_add_event
  fsnotify: Retrieve super block from the data field
  fsnotify: Pass group argument to free_event
  fanotify: Support null inode event in fanotify_dfid_inode
  fanotify: Allow file handle encoding for unhashed events
  fanotify: Encode empty file handle when no inode is provided
  fanotify: Require fid_mode for any non-fd event
  fsnotify: Support FS_ERROR event type
  fanotify: Reserve UAPI bits for FAN_FS_ERROR
  fanotify: Pre-allocate pool of error events
  fanotify: Limit number of marks with FAN_FS_ERROR per group
  fanotify: Support enqueueing of error events
  fanotify: Support merging of error events
  fanotify: Report FID entry even for zero-length file_handle
  fanotify: Report fid info for file related file system errors
  fanotify: Emit generic error info for error event
  fanotify: Allow users to request FAN_FS_ERROR events
  ext4: Send notifications on error
  samples: Add fs error monitoring example
  docs: Document the FAN_FS_ERROR event

 .../admin-guide/filesystem-monitoring.rst     |  76 ++++++++
 Documentation/admin-guide/index.rst           |   1 +
 fs/ext4/super.c                               |   8 +
 fs/notify/fanotify/fanotify.c                 | 122 +++++++++++-
 fs/notify/fanotify/fanotify.h                 |  31 +++-
 fs/notify/fanotify/fanotify_user.c            | 173 ++++++++++++++----
 fs/notify/fsnotify.c                          |   7 +-
 fs/notify/group.c                             |   2 +-
 fs/notify/inotify/inotify_fsnotify.c          |   5 +-
 fs/notify/inotify/inotify_user.c              |   6 +-
 fs/notify/notification.c                      |  14 +-
 include/linux/fanotify.h                      |   9 +-
 include/linux/fsnotify.h                      |  58 ++++--
 include/linux/fsnotify_backend.h              |  96 +++++++++-
 include/uapi/linux/fanotify.h                 |   8 +
 kernel/audit_fsnotify.c                       |   3 +-
 kernel/audit_watch.c                          |   3 +-
 samples/Kconfig                               |   9 +
 samples/Makefile                              |   1 +
 samples/fanotify/Makefile                     |   5 +
 samples/fanotify/fs-monitor.c                 | 142 ++++++++++++++
 21 files changed, 685 insertions(+), 94 deletions(-)
 create mode 100644 Documentation/admin-guide/filesystem-monitoring.rst
 create mode 100644 samples/fanotify/Makefile
 create mode 100644 samples/fanotify/fs-monitor.c

-- 
2.33.0


             reply	other threads:[~2021-10-14 21:37 UTC|newest]

Thread overview: 67+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-10-14 21:36 Gabriel Krisman Bertazi [this message]
2021-10-14 21:36 ` [PATCH v7 01/28] fsnotify: pass data_type to fsnotify_name() Gabriel Krisman Bertazi
2021-10-15  9:18   ` Jan Kara
2021-10-14 21:36 ` [PATCH v7 02/28] fsnotify: pass dentry instead of inode data Gabriel Krisman Bertazi
2021-10-15 13:39   ` Jan Kara
2021-10-18  9:11   ` Jan Kara
2021-10-14 21:36 ` [PATCH v7 03/28] fsnotify: clarify contract for create event hooks Gabriel Krisman Bertazi
2021-10-15  9:21   ` Jan Kara
2021-10-14 21:36 ` [PATCH v7 04/28] fsnotify: Don't insert unmergeable events in hashtable Gabriel Krisman Bertazi
2021-10-14 21:36 ` [PATCH v7 05/28] fanotify: Fold event size calculation to its own function Gabriel Krisman Bertazi
2021-10-14 21:36 ` [PATCH v7 06/28] fanotify: Split fsid check from other fid mode checks Gabriel Krisman Bertazi
2021-10-14 21:36 ` [PATCH v7 07/28] inotify: Don't force FS_IN_IGNORED Gabriel Krisman Bertazi
2021-10-14 21:36 ` [PATCH v7 08/28] fsnotify: Add helper to detect overflow_event Gabriel Krisman Bertazi
2021-10-14 21:36 ` [PATCH v7 09/28] fsnotify: Add wrapper around fsnotify_add_event Gabriel Krisman Bertazi
2021-10-14 21:36 ` [PATCH v7 10/28] fsnotify: Retrieve super block from the data field Gabriel Krisman Bertazi
2021-10-15  5:39   ` Amir Goldstein
2021-10-15  9:26   ` Jan Kara
2021-10-14 21:36 ` [PATCH v7 11/28] fsnotify: Pass group argument to free_event Gabriel Krisman Bertazi
2021-10-15  5:40   ` Amir Goldstein
2021-10-15  9:26   ` Jan Kara
2021-10-14 21:36 ` [PATCH v7 12/28] fanotify: Support null inode event in fanotify_dfid_inode Gabriel Krisman Bertazi
2021-10-15  5:49   ` Amir Goldstein
2021-10-15  9:30   ` Jan Kara
2021-10-14 21:36 ` [PATCH v7 13/28] fanotify: Allow file handle encoding for unhashed events Gabriel Krisman Bertazi
2021-10-14 21:36 ` [PATCH v7 14/28] fanotify: Encode empty file handle when no inode is provided Gabriel Krisman Bertazi
2021-10-15  6:02   ` Amir Goldstein
2021-10-15  9:32   ` Jan Kara
2021-10-14 21:36 ` [PATCH v7 15/28] fanotify: Require fid_mode for any non-fd event Gabriel Krisman Bertazi
2021-10-14 21:36 ` [PATCH v7 16/28] fsnotify: Support FS_ERROR event type Gabriel Krisman Bertazi
2021-10-14 21:36 ` [PATCH v7 17/28] fanotify: Reserve UAPI bits for FAN_FS_ERROR Gabriel Krisman Bertazi
2021-10-15  9:37   ` Jan Kara
2021-10-14 21:36 ` [PATCH v7 18/28] fanotify: Pre-allocate pool of error events Gabriel Krisman Bertazi
2021-10-15  6:19   ` Amir Goldstein
2021-10-15  7:33     ` Amir Goldstein
2021-10-15  9:46   ` Jan Kara
2021-10-14 21:36 ` [PATCH v7 19/28] fanotify: Limit number of marks with FAN_FS_ERROR per group Gabriel Krisman Bertazi
2021-10-15  6:15   ` Amir Goldstein
2021-10-15 16:53     ` Gabriel Krisman Bertazi
2021-10-15 17:49       ` Amir Goldstein
2021-10-14 21:36 ` [PATCH v7 20/28] fanotify: Support enqueueing of error events Gabriel Krisman Bertazi
2021-10-15  7:04   ` Amir Goldstein
2021-10-15 16:50     ` Gabriel Krisman Bertazi
2021-10-15 12:34   ` Jan Kara
2021-10-14 21:36 ` [PATCH v7 21/28] fanotify: Support merging " Gabriel Krisman Bertazi
2021-10-15  7:09   ` Amir Goldstein
2021-10-15 16:54     ` Gabriel Krisman Bertazi
2021-10-15 17:52       ` Amir Goldstein
2021-10-18 13:55         ` Gabriel Krisman Bertazi
2021-10-15 12:43   ` Jan Kara
2021-10-14 21:36 ` [PATCH v7 22/28] fanotify: Report FID entry even for zero-length file_handle Gabriel Krisman Bertazi
2021-10-15  8:10   ` Amir Goldstein
2021-10-15 13:13     ` Jan Kara
2021-10-14 21:36 ` [PATCH v7 23/28] fanotify: Report fid info for file related file system errors Gabriel Krisman Bertazi
2021-10-15  7:56   ` Amir Goldstein
2021-10-15 13:38     ` Jan Kara
2021-10-14 21:36 ` [PATCH v7 24/28] fanotify: Emit generic error info for error event Gabriel Krisman Bertazi
2021-10-15  8:13   ` Amir Goldstein
2021-10-15 12:47   ` Jan Kara
2021-10-14 21:36 ` [PATCH v7 25/28] fanotify: Allow users to request FAN_FS_ERROR events Gabriel Krisman Bertazi
2021-10-15  8:27   ` Amir Goldstein
2021-10-15 12:49   ` Jan Kara
2021-10-14 21:36 ` [PATCH v7 26/28] ext4: Send notifications on error Gabriel Krisman Bertazi
2021-10-14 21:59   ` Theodore Ts'o
2021-10-14 21:36 ` [PATCH v7 27/28] samples: Add fs error monitoring example Gabriel Krisman Bertazi
2021-10-14 21:36 ` [PATCH v7 28/28] docs: Document the FAN_FS_ERROR event Gabriel Krisman Bertazi
2021-10-15  8:38 ` [PATCH v7 00/28] file system-wide error monitoring Amir Goldstein
2021-10-15  9:16 ` Jan Kara

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20211014213646.1139469-1-krisman@collabora.com \
    --to=krisman@collabora.com \
    --cc=amir73il@gmail.com \
    --cc=david@fromorbit.com \
    --cc=dhowells@redhat.com \
    --cc=djwong@kernel.org \
    --cc=jack@suse.com \
    --cc=kernel@collabora.com \
    --cc=khazhy@google.com \
    --cc=linux-api@vger.kernel.org \
    --cc=linux-ext4@vger.kernel.org \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=repnop@google.com \
    --cc=tytso@mit.edu \
    --subject='Re: [PATCH v7 00/28] file system-wide error monitoring' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).