Re: Make note information human readable when help -D

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hello Dave,

Thank you for suggestions.
I think I have further understanding of "keep it simple".

This time, there are five functions in the patch.
They are:
	display_ELF_note()
	display_prstatus_x86_64()
	display_prstatus_x86()
	display_qemu_x86_64()
	display_qemu_x86()

It looks much better.
Previously, I tried to avoid duplicate code though it can bring
more complexity. Now, I think simple in logic is more important.
I do appreciate your teaching me so much.

On 12/12/2014 01:06 AM, Dave Anderson wrote:


----- Original Message -----
On 12/11/2014 06:27 AM, Dave Anderson wrote:

First, please address all of these warnings:

$ make warn
... [ cut ] ...
cc -c -g -DX86_64 -DLZO -DSNAPPY -DGDB_7_6  netdump.c -Wall -O2
-Wstrict-prototypes -Wmissing-prototypes -fstack-protector
-Wformat-security
netdump.c: In function 'dump_Elf32_Nhdr':
netdump.c:1987:4: warning: format not a string literal and no format
arguments [-Wformat-security]
netdump.c: In function 'dump_Elf64_Nhdr':
netdump.c:2272:4: warning: format not a string literal and no format
arguments [-Wformat-security]
netdump.c:2303:4: warning: format not a string literal and no format
arguments [-Wformat-security]
...


Hello Dave,

These warnings have been addressed.

Secondly, for compressed kdumps in diskdump.c, you have this construct:

         if (dd->machine_type == EM_386)
                 display_note_elf32(dd->nt_prstatus_percpu[i],l_buf);
         else if (dd->machine_type == EM_X86_64)
                 display_note_elf64(dd->nt_prstatus_percpu[i],l_buf);

But for ELF kdumps in netdump.c, display_note_elf32() are display_note_elf64()
look to be called unconditionally.  What about the other architectures?


I distinguish the architectures by the struct size in display_note(). I wonder
if it is needed to add other conditions like machine_type.


	display_note(void *note_ptr, char *buf, int descsz)
	{
          	if (descsz == (2 * sizeof(struct x86_64_prstatus)))
                  	display_prstatus_elf64(note_ptr, buf);
	        else if (descsz == sizeof(struct x86_prstatus))
          	        display_prstatus_elf32(note_ptr, buf);
	        else if (descsz == (2 * sizeof(QEMUCPUState)))
          	        display_qemu_elf64(note_ptr, buf);
	        else if (descsz == sizeof(QEMUCPUState))
	                display_qemu_elf32(note_ptr, buf);
	}

Yes, it should definitely take the machine type into account, given
that it's not guaranteed to work with any other architectures without
knowing exactly what the size of their notes would be.

In fact this whole patchset needs to be made more machine-type specific.
The myriad of new functions you've added to netdump.c are generally confusing
to me because several of their names imply that they are generic to ELF,
and not specific to x86 and x86_64.  Yes, the data being displayed is contained
within an ELF note, but more importantly, the data itself is machine-specific.

So can we please simplify/clarify things a bit?

Here's what I suggest.  Given the fact that display_prstatus_el64()
and display_prstatus_el32 are coming from two different dumpfile locations
(and perhaps more in the future?), why not add machine type and note type
arguments?  To reduce the number of functions, create a general purpose
"display_ELF_note()" function that can distribute the work appropriately?

So diskdump.c can be simplified from:

         if (dd->machine_type == EM_386)
                 display_note_elf32(dd->nt_prstatus_percpu[i],l_buf);
         else if (dd->machine_type == EM_X86_64)
                 display_note_elf64(dd->nt_prstatus_percpu[i],l_buf);
to just:

         display_ELF_note(dd->machine_type, PRSTATUS, dd->nt_prstatus_percpu[i], l_buf);

And the 3 netdump.c invocations would be changed from:

         display_note_elf32(note, l_buf);
         display_note_elf32(note, l_buf);
         display_note_el64((note, l_buf);

to:
         display_ELF_note(EM_386, qemuinfo ? QEMU : PRSTATUS, note, l_buf);
         display_ELF_note(EM_386, PRSTATUS, note, l_buf);
         display_ELF_note(EM_X86_64, qemuinfo ? QEMU : PRSTATUS, note, l_buf);

Then the general-purpose display_ELF_note() function can directly call
the appropriate function.  And those architecture-specific functions
should be should be renamed to append "_x86" or "_x86_64" (as you have
done with "x86_prstatus" and "x86_64_prstatus" structures).

So the display_ELF_note() function could look something like:

void
display_ELF_note(int machine, int type, void *note, char *buf)
{
         switch (machine)
         {
         case EM_386:
                 switch (type)
                 {
                 case PRSTATUS:
                        display_prstatus_x86(note, buf);
                        break;
                 case QEMU:
                        display_qemu_x86(note, buf);
                        break;
                 }
                 break;

         case EM_X86_64:
                 switch (type)
                 {
                 case PRSTATUS:
                        display_prstatus_x86_64(note, buf);
                        break;
                 case QEMU:
                        display_qemu_x86_64(note, buf);
                        break;
                 }
                 break;

         default:
                 return;
         }
}

And in the future, functions can simply be added for other architectures.

Now, I understand that your display_qemu_elf32() and display_qemu_elf64()
functions call into your single display_qemu_elf() function because there
is common code.  That could be handled in few different manners.

You could consolidate my suggested display_qemu_x86() and display_qemu_x86_64()
functions into a "common" function that is named display_qemu_x86_32_64(), or
display_qemu_intel(), or whatever (?).

Alternatively, you could have display_ELF_note() call an architecture-neutral
display_qemu() function, which can set up things based upon the machine-type,
and then call the architecture-specific function -- which in this case would
be the common function used by both x86 and x86_64.

Or your display_qemu_elf() function could be broken into two separate
functions, which would contain some common/duplicate code.  That's also
fine with me.

However you do it, all I ask is that if a function only applies to a specific
architecture, then please follow the structure-naming convention so that there
is no question.

Thanks,
   Dave




--
Thanks
Zhou Wenjian
From d860097c5cd3da21eb9008298d3d290b527f1437 Mon Sep 17 00:00:00 2001
From: Zhou Wenjian <zhouwj-fnst@xxxxxxxxxxxxxx>
Date: Fri, 12 Dec 2014 11:07:52 +0800
Subject: [PATCH] Make note info human readable when help -D

---
 defs.h     |    6 +
 diskdump.c |   10 ++
 netdump.c  |  313 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 329 insertions(+), 0 deletions(-)

diff --git a/defs.h b/defs.h
index 2e52bc4..5af2534 100755
--- a/defs.h
+++ b/defs.h
@@ -6117,3 +6117,9 @@ extern int have_full_symbols(void);
 #endif
 
 #endif /* !GDB_COMMON */
+
+/*
+ *  for display_ELF_note() type
+ */
+#define PRSTATUS	0x1
+#define QEMU		0X2
diff --git a/diskdump.c b/diskdump.c
index 3d33fdc..da8d81a 100644
--- a/diskdump.c
+++ b/diskdump.c
@@ -83,6 +83,8 @@ static char *vmcoreinfo_read_string(const char *);
 static void diskdump_get_osrelease(void);
 static int valid_note_address(unsigned char *);
 
+void display_ELF_note(int, int, void *, char *);
+
 /* For split dumpfile */
 static struct diskdump_data **dd_list = NULL;
 static int num_dd = 0;
@@ -1736,10 +1738,18 @@ __diskdump_memory_dump(FILE *fp)
 				dd->num_prstatus_notes);
 			fprintf(fp, "           notes_buf: %lx\n",
 				(ulong)dd->notes_buf);
+
+			char *l_buf = (char *)malloc(2 * BUFSIZE);
 			for (i = 0; i < dd->num_prstatus_notes; i++) {
 				fprintf(fp, "            notes[%d]: %lx\n",
 					i, (ulong)dd->nt_prstatus_percpu[i]);
+
+				display_ELF_note(dd->machine_type, PRSTATUS,
+						 dd->nt_prstatus_percpu[i],l_buf);
+				fprintf(fp, "%s", l_buf);
+				memset(l_buf, 0, 2 * BUFSIZE);
 			}
+			free(l_buf);
 			dump_nt_prstatus_offset(fp);
 		}
 		if (dh->header_version >= 5) {
diff --git a/netdump.c b/netdump.c
index 903faa0..258156b 100644
--- a/netdump.c
+++ b/netdump.c
@@ -45,6 +45,7 @@ static int proc_kcore_init_64(FILE *fp);
 static char *get_regs_from_note(char *, ulong *, ulong *);
 static void kdump_get_osrelease(void);
 static char *vmcoreinfo_read_string(const char *);
+void display_ELF_note(int, int, void *, char *);
 
 
 #define ELFSTORE 1
@@ -1976,6 +1977,12 @@ dump_Elf32_Nhdr(Elf32_Off offset, int store)
                 }
                 lf = 0;
 	} else {
+		if (nd->ofp) {
+			char *l_buf = (char *)malloc(2 * BUFSIZE);
+			display_ELF_note(EM_386, qemuinfo ? QEMU : PRSTATUS, note,l_buf);
+			fprintf(fp, "%s", l_buf);
+			free(l_buf);
+		}
 		for (i = lf = 0; i < note->n_descsz/sizeof(ulong); i++) {
 			if (((i%4)==0)) {
 				netdump_print("%s                         ", 
@@ -2255,6 +2262,13 @@ dump_Elf64_Nhdr(Elf64_Off offset, int store)
 	}
 
 	if (BITS32() && (xen_core || (note->n_type == NT_PRSTATUS))) {
+		if (!xen_core && nd->ofp) {
+			char *l_buf = (char *)malloc(2 * BUFSIZE);
+			display_ELF_note(EM_386, PRSTATUS, note, l_buf);
+			fprintf(fp, "%s", l_buf);
+			free(l_buf);
+		}
+
 		iptr = (int *)uptr;
 		for (i = lf = 0; i < note->n_descsz/sizeof(ulong); i++) {
 			if (((i%4)==0)) {
@@ -2279,6 +2293,12 @@ dump_Elf64_Nhdr(Elf64_Off offset, int store)
 		iptr = (int *)uptr;
 		netdump_print("                         %08lx\n", *iptr); 
 	} else {
+		if (nd->ofp) {
+			char *l_buf = (char *)malloc(2 * BUFSIZE);
+			display_ELF_note(EM_X86_64, qemuinfo ? QEMU : PRSTATUS, note, l_buf);
+			fprintf(fp, "%s", l_buf);
+			free(l_buf);
+		}
 		for (i = lf = 0; i < note->n_descsz/sizeof(ulonglong); i++) {
 			if (((i%2)==0)) {
 				netdump_print("%s                         ", 
@@ -2543,6 +2563,299 @@ struct x86_64_user_regs_struct {
         unsigned long ds,es,fs,gs;
 };
 
+struct x86_64_prstatus {
+	int si_signo;
+	int si_code;
+	int si_errno;
+	short cursig;
+	unsigned long sigpend;
+	unsigned long sighold;
+	int pid;
+	int ppid;
+	int pgrp;
+	int sid;
+	struct timeval utime;
+	struct timeval stime;
+	struct timeval cutime;
+	struct timeval cstime;
+	struct x86_64_user_regs_struct regs;
+	int fpvalid;
+};
+
+static void
+display_prstatus_x86_64(void *note_ptr, char *buf)
+{
+	struct x86_64_prstatus *pr;
+	Elf64_Nhdr *note;
+
+	note = (Elf64_Nhdr *)note_ptr;
+	pr = (struct x86_64_prstatus *)(
+		(char *)note + sizeof(Elf64_Nhdr) + note->n_namesz);
+	pr = (struct x86_64_prstatus *)roundup((ulong)pr, 4);
+	sprintf(buf,
+		"\t\tsi.signo: %d, si.code: %d, si.errno: %d, cursig:%d\n"
+		"\t\tsigpend: %lu\n"
+		"\t\tsighold: %lu\n"
+		"\t\tpid: %d, ppid: %d, pgrp: %d, sid:%d\n"
+		"\t\tutime: %01lld.%06d, stime: %01lld.%06d, cutime: %01lld.%06d, "
+		"cstime: %01lld.%06d\n"
+		"\t\tORIG_RAX: %lu, fpvalid: %d\n"
+		"\t\tR15    : 0x%016lx\tR14     : 0x%016lx\n"
+		"\t\tR13    : 0x%016lx\tR12     : 0x%016lx\n"
+		"\t\tRBP    : 0x%016lx\tRBX     : 0x%016lx\n"
+		"\t\tR11    : 0x%016lx\tR10     : 0x%016lx\n"
+		"\t\tR9     : 0x%016lx\tR8      : 0x%016lx\n"
+		"\t\tRAX    : 0x%016lx\tRCX     : 0x%016lx\n"
+		"\t\tRDX    : 0x%016lx\tRSI     : 0x%016lx\n"
+		"\t\tRDI    : 0x%016lx\tRIP     : 0x%016lx\n"
+		"\t\tRFLAGS : 0x%016lx\tRSP     : 0x%016lx\n"
+		"\t\tFS_BASE: 0x%016lx\tGS_BASE : 0x%016lx\n"
+		"\t\tCS: 0x%04lx SS: 0x%04lx DS: 0x%04lx ES: 0x%04lx "
+		"FS: 0x%04lx GS: 0x%04lx\n",
+		pr->si_signo, pr->si_code, pr->si_errno, pr->cursig,
+		pr->sigpend, pr->sighold,
+		pr->pid, pr->ppid, pr->pgrp, pr->sid,
+		(long long)pr->utime.tv_sec, (int)pr->utime.tv_usec,
+		(long long)pr->stime.tv_sec, (int)pr->stime.tv_usec,
+		(long long)pr->cutime.tv_sec, (int)pr->cutime.tv_usec,
+		(long long)pr->cstime.tv_sec, (int)pr->cstime.tv_usec,
+		pr->regs.orig_rax, pr->fpvalid,
+		pr->regs.r15, pr->regs.r14,
+		pr->regs.r13, pr->regs.r12,
+		pr->regs.rbp, pr->regs.rbx,
+		pr->regs.r11, pr->regs.r10,
+		pr->regs.r9, pr->regs.r8,
+		pr->regs.rax, pr->regs.rcx,
+		pr->regs.rdx, pr->regs.rsi,
+		pr->regs.rdi, pr->regs.rip,
+		pr->regs.eflags, pr->regs.rsp,
+		pr->regs.fs_base, pr->regs.gs_base,
+		pr->regs.cs, pr->regs.ss,
+		pr->regs.ds, pr->regs.es,
+		pr->regs.fs, pr->regs.gs
+		);
+}
+
+struct x86_user_regs_struct {
+	unsigned long ebx,ecx,edx,esi,edi,ebp,eax;
+	unsigned long ds,es,fs,gs,orig_eax;
+	unsigned long eip,cs,eflags;
+	unsigned long esp,ss;
+};
+
+struct x86_prstatus {
+	int si_signo;
+	int si_code;
+	int si_errno;
+	short cursig;
+	unsigned long sigpend;
+	unsigned long sighold;
+	int pid;
+	int ppid;
+	int pgrp;
+	int sid;
+	struct timeval utime;
+	struct timeval stime;
+	struct timeval cutime;
+	struct timeval cstime;
+	struct x86_user_regs_struct regs;
+	int fpvalid;
+};
+
+static void
+display_prstatus_x86(void *note_ptr, char *buf)
+{
+	struct x86_prstatus *pr;
+	Elf32_Nhdr *note;
+
+	note = (Elf32_Nhdr *)note_ptr;
+	pr = (struct x86_prstatus *)(
+		(char *)note + sizeof(Elf32_Nhdr) + note->n_namesz);
+	pr = (struct x86_prstatus *)roundup((ulong)pr, 4);
+
+	sprintf(buf,
+		"\t\tsi.signo: %d si.code: %d si.errno: %d cursig: %d\n"
+		"\t\tsigpend : %lu\n"
+		"\t\tsighold : %lu\n"
+		"\t\tpid: %d ppid: %d pgrp: %d sid: %d\n"
+		"\t\tutime: %01lld.%06d, stime: %01lld.%06d, cutime: %01lld.%06d, "
+		"cstime: %01lld.%06d\n"
+		"\t\torig_eax: %lu, fpvalid: %d\n"
+		"\t\tEBX    : 0x%08lx\tECX     : 0x%08lx\n"
+		"\t\tEDX    : 0x%08lx\tESI     : 0x%08lx\n"
+		"\t\tEDI    : 0x%08lx\tEBP     : 0x%08lx\n"
+		"\t\tEAX    : 0x%08lx\tEIP     : 0x%08lx\n"
+		"\t\tEFLAGS : 0x%08lx\tESP     : 0x%08lx\n"
+		"\t\tDS: 0x%04lx ES: 0x%04lx FS: 0x%04lx GS: 0x%04lx "
+		"CS: 0x%04lx SS: 0x%04lx\n",
+		pr->si_signo, pr->si_code, pr->si_errno, pr->cursig,
+		pr->sigpend, pr->sighold,
+		pr->pid, pr->ppid, pr->pgrp, pr->sid,
+		(long long)pr->utime.tv_sec, (int)pr->utime.tv_usec,
+		(long long)pr->stime.tv_sec, (int)pr->stime.tv_usec,
+		(long long)pr->cutime.tv_sec, (int)pr->cutime.tv_usec,
+		(long long)pr->cstime.tv_sec, (int)pr->cstime.tv_usec,
+		pr->regs.orig_eax, pr->fpvalid,
+		pr->regs.ebx, pr->regs.ecx,
+		pr->regs.edx, pr->regs.esi,
+		pr->regs.edi, pr->regs.ebp,
+		pr->regs.eax, pr->regs.eip,
+		pr->regs.eflags, pr->regs.esp,
+		pr->regs.ds, pr->regs.es, pr->regs.fs,
+		pr->regs.gs, pr->regs.cs, pr->regs.ss
+		);
+}
+
+static void
+display_qemu_x86_64(void *note_ptr, char *buf)
+{
+	int i, size;
+	Elf64_Nhdr *note;
+	QEMUCPUState *ptr;
+	QEMUCPUSegment *seg;
+	char *seg_names[] = {"CS", "DS", "ES", "FS", "GS", "SS", "LDT", "TR",
+			     "GDT", "IDT"};
+
+	note = (Elf64_Nhdr *)note_ptr;
+	ptr = (QEMUCPUState *)(
+		(char *)note + sizeof(Elf64_Nhdr) + note->n_namesz);
+
+	ptr = (QEMUCPUState *)roundup((ulong)ptr, 4);
+	seg = &(ptr->cs);
+
+	size = sprintf(buf,
+			"\t\tversion: 0x%08x\tsize: 0x%08x\n"
+			"\t\tRAX   : 0x%016llx\tRBX   : 0x%016llx\n"
+			"\t\tRCX   : 0x%016llx\tRDX   : 0x%016llx\n"
+			"\t\tRSI   : 0x%016llx\tRDI   : 0x%016llx\n"
+			"\t\tRSP   : 0x%016llx\tRBP   : 0x%016llx\n"
+			"\t\tRIP   : 0x%016llx\tRFLAGS: 0x%016llx\n"
+			"\t\tR8    : 0x%016llx\tR9    : 0x%016llx\n"
+			"\t\tR10   : 0x%016llx\tR11   : 0x%016llx\n"
+			"\t\tR12   : 0x%016llx\tR13   : 0x%016llx\n"
+			"\t\tR14   : 0x%016llx\tR15   : 0x%016llx\n",
+			ptr->version, ptr->size,
+			(ulonglong)ptr->rax, (ulonglong)ptr->rbx,
+			(ulonglong)ptr->rcx, (ulonglong)ptr->rdx,
+			(ulonglong)ptr->rsi, (ulonglong)ptr->rdi,
+			(ulonglong)ptr->rsp, (ulonglong)ptr->rbp,
+			(ulonglong)ptr->rip, (ulonglong)ptr->rflags,
+			(ulonglong)ptr->r8, (ulonglong)ptr->r9,
+			(ulonglong)ptr->r10, (ulonglong)ptr->r11,
+			(ulonglong)ptr->r12, (ulonglong)ptr->r13,
+			(ulonglong)ptr->r14, (ulonglong)ptr->r15
+			);
+		buf += size;
+
+	for(i = 0; i < sizeof(seg_names)/sizeof(seg_names[0]); i++) {
+		size = sprintf(buf,
+				"\t\t%s:\n"
+				"\t\tselector: 0x%08x\tlimit: 0x%08x\tflags: 0x%08x\n"
+				"\t\tpad     : 0x%08x\tbase : 0x%016llx\n",
+				seg_names[i],
+				seg->selector, seg->limit, seg->flags,
+				seg->pad, (ulonglong)seg->base
+				);
+		buf += size;
+		seg++;
+	}
+
+	sprintf(buf,
+		"\t\tcr[0]: %016llx\tcr[1]: %016llx\tcr[2]: %016llx\n"
+		"\t\tcr[3]: %016llx\tcr[4]: %016llx\n",
+		(ulonglong)ptr->cr[0], (ulonglong)ptr->cr[1], (ulonglong)ptr->cr[2],
+		(ulonglong)ptr->cr[3], (ulonglong)ptr->cr[4]
+		);
+}
+
+static void
+display_qemu_x86(void *note_ptr, char *buf)
+{
+	int i, size, t=0;
+	Elf32_Nhdr *note;
+	QEMUCPUState *ptr;
+	QEMUCPUSegment *seg;
+	char *seg_names[] = {"CS", "DS", "ES", "FS", "GS", "SS", "LDT", "TR",
+			     "GDT", "IDT"};
+
+
+	note = (Elf32_Nhdr *)note_ptr;
+	ptr = (QEMUCPUState *)(
+		(char *)note + sizeof(Elf32_Nhdr) + note->n_namesz);
+	ptr = (QEMUCPUState *)roundup((ulong)ptr, 4);
+	seg = &(ptr->cs);
+
+	size = sprintf(buf,
+			"\t\tversion: 0x%08x\tsize: 0x%08x\n"
+			"\t\tEAX   : 0x%016llx\tEBX   : 0x%016llx\n"
+			"\t\tECX   : 0x%016llx\tEDX   : 0x%016llx\n"
+			"\t\tESI   : 0x%016llx\tEDI   : 0x%016llx\n"
+			"\t\tESP   : 0x%016llx\tEBP   : 0x%016llx\n"
+			"\t\tEIP   : 0x%016llx\tEFLAGS: 0x%016llx\n",
+			ptr->version, ptr->size,
+			(ulonglong)ptr->rax, (ulonglong)ptr->rbx, (ulonglong)ptr->rcx,
+			(ulonglong)ptr->rdx, (ulonglong)ptr->rsi, (ulonglong)ptr->rdi,
+			(ulonglong)ptr->rsp, (ulonglong)ptr->rbp,
+			(ulonglong)ptr->rip, (ulonglong)ptr->rflags
+			);
+	buf += size;
+	t+=size;
+
+	for(i = 0; i < sizeof(seg_names)/sizeof(seg_names[0]); i++) {
+		size = sprintf(buf,
+				"\t\t%s:\n"
+				"\t\tselector: 0x%08x\tlimit: 0x%08x\tflags: 0x%08x\n"
+				"\t\tpad     : 0x%08x\tbase : 0x%016llx\n",
+				seg_names[i],
+				seg->selector, seg->limit, seg->flags,
+				seg->pad, (ulonglong)seg->base
+				);
+		buf += size;
+		seg++;
+	}
+
+	sprintf(buf,
+		"\t\tcr[0]: %016llx\tcr[1]: %016llx\tcr[2]: %016llx\n"
+		"\t\tcr[3]: %016llx\tcr[4]: %016llx\n",
+		(ulonglong)ptr->cr[0], (ulonglong)ptr->cr[1], (ulonglong)ptr->cr[2],
+		(ulonglong)ptr->cr[3], (ulonglong)ptr->cr[4]
+		);
+}
+
+void
+display_ELF_note(int machine, int type, void *note, char *buf)
+{
+	switch (machine)
+	{
+	case EM_386:
+		switch (type)
+		{
+		case PRSTATUS:
+			display_prstatus_x86(note, buf);
+			break;
+		case QEMU:
+			display_qemu_x86(note, buf);
+			break;
+		}
+		break;
+
+	case EM_X86_64:
+		switch (type)
+		{
+		case PRSTATUS:
+			display_prstatus_x86_64(note, buf);
+			break;
+		case QEMU:
+			display_qemu_x86_64(note, buf);
+			break;
+		}
+		break;
+
+	default:
+		return;
+	}
+}
+
 void 
 get_netdump_regs_x86_64(struct bt_info *bt, ulong *ripp, ulong *rspp)
 {
-- 
1.7.1

--
Crash-utility mailing list
Crash-utility@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/crash-utility

[Index of Archives]     [Fedora Development]     [Fedora Desktop]     [Fedora SELinux]     [Yosemite News]     [KDE Users]     [Fedora Tools]

 

Powered by Linux