Problems accessing files with gvfs-fuse-daemon

Post by Gustau PÃ©rez
Hi everyone,
I'm having problems with gvfs-fuse-daemon. I'm trying to find out why
apps like vlc are unable to read files
in a samba share. As you may know, if you try to open a file in a share
throu nautilus, if the apps is not gvfs-aware nautilus launches the apps
$HOME/.gvfs/share in server/file.avi
then through it works through the gvfs-fuse-daemon which does the magic.
Well, anytime I try this, the app seems to freeze. No matter which
# cp $HOME/.gvfs/share in server/file.avi /tmp
Then, with tcpdump I saw a few packets were transferred between the
server and the client then transfer stopped. In the terminal I got and
Input/output error.
Even the gvfs-* apps (like gvfs-cat, gvfs-copy) fail. The error with
this apps comes from glib (gio part, when making a read syscall, so it
makes me think that probably the error is in gvfs-fuse-daemon, when the
kernel returns the read syscall to the fuse daemon that handles the
reads in that mount point).
I noticed that when I let gnome-session launch gvfs-fuse-daemon
without arguments (normal situation) the transfer stops at 1Mb of size.
If I try launching gvfs-fuse-daemon with "-o direct_io" in a
different mount like (say .gvfs_tmp), then I'm able
to copy files with both cp and gvfs-copy through that mount point. Even
I'm able to play mp3 files with mpg123, but vlc, totem and other apps
still fail to access that share (they transfer a little amount of data
and then they stop).
With this config (with .gvfs_tmp, -o direct_io), If I try to launch
and mpg123 (which works) and then and vlc instance, everything freezes
again. mpg123 stops playing until I kill vlc, then it can continue
playing. The same with kaffeine and other friends not gvfs-aware.
As I said, if I let gvfs-fuse-daemon without options (as
gnome-session does) I can do nothing through .gvfs/. No cp, no mpg123,
anything at all. -o direct_io allows me a few things.
I'm yesterday updated (before the jpeg library update). gvfs is 1.4.3_1.
FreeBSD gusiport 8.0-STABLE FreeBSD 8.0-STABLE #0: Fri Feb 5
18:50:54 CET 2010
Clean /etc/src.conf (no special options given there). The fuse
module is update before the kernel and world were last updated.
I'm puzzled because I saw anyone having such a problem. I checked
/usr/ports/UPDATING, just I case I missed anything importa.
Is anyone having such a problem ? Any idea where can I start
debugging ? If I can provide any additional info, let me know.

The fact that direct_io kind of works leads me to think this a problem
with fuse, and not gvfs. Gvfs will simply call fuse_main() with the
arguments passed to it. It's up to the fuse userland and kernel module
to provide a lot of the heavy lifting. To that end, you might try
rebuilding your kernel module, or ask the fuse maintainer if they have
some ideas.

Joe

--
PGP Key : http://www.marcuscom.com/pgp.asc

Gustau Pérez

2010-02-07 20:07:07 UTC

I debugged the daemon with truss. I got a very strange serie of
syscall with a lot of POLLs and READs to a channel returning "ERR#35
resource temporary unavailable). I discovered some socket call
returning a channel that then produce that read errors. Any idea ?

I posted the trace to pastebin :

http://pastebin.com/m79cf37f3

If you search for socket syscalls, you'll see it normally creates two,
and then it connects them to a unix socket in /var/tmp. For example I
see it connecting two sockets to

/var/tmp/gvfs-gus-B9lo36Ky/socket1
/var/tmp/gvfs-gus-B9lo36Ky/socket2

But only the second can be found. I can't see the first. So probably
this is the problem. Anyone has the same problem ? I can provide more info.

Regards,

Gus

--
PGP KEY : http://www-entel.upc.edu/gus/gus.asc

Joe Marcus Clarke

2010-02-07 20:17:41 UTC

I debugged the daemon with truss. I got a very strange serie of
syscall with a lot of POLLs and READs to a channel returning "ERR#35
resource temporary unavailable). I discovered some socket call
returning a channel that then produce that read errors. Any idea ?
http://pastebin.com/m79cf37f3
If you search for socket syscalls, you'll see it normally creates two,
and then it connects them to a unix socket in /var/tmp. For example I
see it connecting two sockets to
/var/tmp/gvfs-gus-B9lo36Ky/socket1
/var/tmp/gvfs-gus-B9lo36Ky/socket2
But only the second can be found. I can't see the first. So probably
this is the problem. Anyone has the same problem ? I can provide more info.

As I said, you would be better off contacting the fuse maintainer.

Joe

--
PGP Key : http://www.marcuscom.com/pgp.asc

Gustau Pérez

2010-02-07 20:25:23 UTC

Post by Gustau PÃ©rez
http://pastebin.com/m79cf37f3
If you search for socket syscalls, you'll see it normally creates two,
and then it connects them to a unix socket in /var/tmp. For example I
see it connecting two sockets to
/var/tmp/gvfs-gus-B9lo36Ky/socket1
/var/tmp/gvfs-gus-B9lo36Ky/socket2
But only the second can be found. I can't see the first. So probably
this is the problem. Anyone has the same problem ? I can provide more info.

As I said, you would be better off contacting the fuse maintainer.
Joe

Hi again,

Well, I think those files had been openened by gvfs-fuse-daemon, I
see gvfs-fuse-daemon opening them in the trace file. So I'm not sure
whether it is fuse responsability or not. Am I right ?

If you still think it is fuse responsability I'll contact the fuse
mantainer, but it is kinda strange fusefs-ssh has not that problem in my
config.

Do you guys have the same problem accessing $HOME/.gvfs files ?

Thanks Joe,

Gus

--
PGP KEY : http://www-entel.upc.edu/gus/gus.asc

Joe Marcus Clarke

2010-02-07 20:29:45 UTC

As I said, you would be better off contacting the fuse maintainer.
Joe

gvfs-fuse-daemon is nothing more than a fuse client. The daemon
initializes the fuse subsystem, then it's up to fuse to invoke callbacks
in gvfs-fuse-daemon. Look at the code in gvfs' client/gvfsfusedaemon.c
for more details.

Post by Gustau PÃ©rez
If you still think it is fuse responsability I'll contact the fuse
mantainer, but it is kinda strange fusefs-ssh has not that problem in my
config.
Do you guys have the same problem accessing $HOME/.gvfs files ?

I don't use fuse.

Joe

--
PGP Key : http://www.marcuscom.com/pgp.asc

Joe Marcus Clarke

2010-02-07 20:49:30 UTC

As I said, you would be better off contacting the fuse maintainer.
Joe

I've been debugging gvfsfusedaemon.c adding some printf's to the
vfs_read callback function, cause first
I though it was a locking problem. gvfsfusedaemon uses a lock to protect
the access when reading a given file. With these printf's I saw that was
not the problem.
So it could be a problem somewhere else in gvfsfusedaemon or maybe in
fuse (I suppose in fusefs-libs not fusefs-kmod). I think the hint is
those pair of sockets it creates. I see it creates some sockets to dbus
in gvfsdaemondbus.c, may be this is the source of the problem.

Look at what direct_io does, and start there. What is different when
that option is specified? That option is never passed to the gvfs code,
so where in the fuse code does it make an impact?

Joe

What I don't understand is why fusefs-ssh is working fine,
transfering media for a long time without problems. Something doesn't
work somewhere, but i don't know what.
Thanks again,
Gus

I don't use fuse.
Joe

--
PGP Key : http://www.marcuscom.com/pgp.asc

Gustau Pérez

2010-02-07 21:26:25 UTC

As I said, you would be better off contacting the fuse maintainer.
Joe

Look at what direct_io does, and start there. What is different when
that option is specified? That option is never passed to the gvfs code,
so where in the fuse code does it make an impact?
Joe

Well, in fusefs-libs I see I changes the file operations if the
direct_io option is passed. But for me fusefs is a new world, and I
don't know how the pieces fit in fusefs.

So I think I would be easier to first check gvfsdaemondbus and
gvfsfusedaemon. Direct_io seems to have some impact, but the problem
remains there when using a no-simple applications (for example vlc and
totem make the problem arise). The pair of sockets problem makes me
think direct_io solves something, but there is another problem.

And I remember pcbsd opening samba shares and vlc and gimp opening
files in those shares through dolphin. I don't know if kde4 and its kios
use fuse or not. I'll check it.

I'll contact the fusefs mantainer, maybe he has any idea, but if
anyone has some spare time and want to have fun ...

Regards,

Gus

--
PGP KEY : http://www-entel.upc.edu/gus/gus.asc

Gustau Pérez

2010-02-13 17:26:35 UTC

Hello again,

Post by Joe Marcus Clarke
Look at what direct_io does, and start there. What is different when
that option is specified? That option is never passed to the gvfs code,
so where in the fuse code does it make an impact?
Joe

tried to contact fusefs-{libs|kmod} but I still got no answer. In
the meantime, I tried to follow the execution of a request, and
found where it gets stuck.

I tried to discover what direct_io does. But fusefs-{libs|kmod} is a
new world for me, so I decided to investigate gvfs first.

It seems that everything fails in function "static gint
gettatr_for_file", in the $WRKDIR/client/gvfsfusedaemon.c when calling
g_file_query_info (which is part of the glib20). To be exact, it asks
for a number of attributes, one of them is
"G_FILE_ATTRIBUTE_STANDARD_SIZE".

If I remove this attribute from the g_file_query_info call,
gvfsfusedaemon doesn't get stuck anymore. The problem is that no app
will be able to use file mounted with gvfs, as all the apps want to know
the size of the files they are working with.

So it seems gvfsfusedaemon does not get stuck, but it's me who does
not know how to solve the problem, because I don't know
how "g_file_query_info" gets info about a file in $HOME/.gvfs/... I
think that when g_file_query_info is called (in glib) it
should in turn try to stat the file, and should be done through fuse,
which in turn will call again gvfsfusedaemon for the info.

I don't know whether I'm right or not. And in if I'm right, I don't
how to move forward. Does anyone have an idea how to continue ?

Regards,

Gus

--
PGP KEY : http://www-entel.upc.edu/gus/gus.asc

Joe Marcus Clarke

2010-02-13 18:31:04 UTC

tried to contact fusefs-{libs|kmod} but I still got no answer. In
the meantime, I tried to follow the execution of a request, and
found where it gets stuck.
I tried to discover what direct_io does. But fusefs-{libs|kmod} is a
new world for me, so I decided to investigate gvfs first.
It seems that everything fails in function "static gint
gettatr_for_file", in the $WRKDIR/client/gvfsfusedaemon.c when calling
g_file_query_info (which is part of the glib20). To be exact, it asks
for a number of attributes, one of them is
"G_FILE_ATTRIBUTE_STANDARD_SIZE".
If I remove this attribute from the g_file_query_info call,
gvfsfusedaemon doesn't get stuck anymore. The problem is that no app
will be able to use file mounted with gvfs, as all the apps want to know
the size of the files they are working with.
So it seems gvfsfusedaemon does not get stuck, but it's me who does
not know how to solve the problem, because I don't know
how "g_file_query_info" gets info about a file in $HOME/.gvfs/... I
think that when g_file_query_info is called (in glib) it
should in turn try to stat the file, and should be done through fuse,
which in turn will call again gvfsfusedaemon for the info.
I don't know whether I'm right or not. And in if I'm right, I don't
how to move forward. Does anyone have an idea how to continue ?

This sounds right. Chances are things are locking up somewhere in the
kernel. You should first try breaking into the hung application (and
gvfs-fuse-daemon) with gdb, and get an idea of where in the code path
it's hanging. I'm betting you will then need to break into the kernel
debugger, and see where in the kernel things get stuck.

Joe

--
PGP Key : http://www.marcuscom.com/pgp.asc

Gustau Pérez

2010-02-07 20:44:29 UTC

As I said, you would be better off contacting the fuse maintainer.
Joe

I've been debugging gvfsfusedaemon.c adding some printf's to the
vfs_read callback function, cause first
I though it was a locking problem. gvfsfusedaemon uses a lock to protect
the access when reading a given file. With these printf's I saw that was
not the problem.

So it could be a problem somewhere else in gvfsfusedaemon or maybe in
fuse (I suppose in fusefs-libs not fusefs-kmod). I think the hint is
those pair of sockets it creates. I see it creates some sockets to dbus
in gvfsdaemondbus.c, may be this is the source of the problem.

What I don't understand is why fusefs-ssh is working fine,
transfering media for a long time without problems. Something doesn't
work somewhere, but i don't know what.

Thanks again,

Gus